apache/cloudstack
1
0
Fork 0
mirror of https://github.com/apache/cloudstack.git synced 2025-11-02 20:02:29 +01:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #731 from remibergsma/centos7-kvm

Browse Source 
CLOUDSTACK-8443: Support CentOS 7 as KVM hypervisorThis adds support for CentOS 7 as a KVM hypervisor. As discussed in the Jira issue, the root cause of the issue was the `cloudstack-setup-agent` script that added this line to `/etc/libvirt/qemu.conf`:

`cgroup_controllers=["cpu"]`

On CentOS 6 this 'works', but since by default no cgroups are mounted, this setting is ignored by `libvirtd`. On CentOS 7 is does not work, because the `cpu` and `cpuacct` are 'co-mounted'. This simpy means you cannot use one without the order. The config line above instructs `libvirtd` to only use the `cpu` cgroup, which it can't.

There was code added to make sure the `cpu` cgroup was not co-mounted. It did this with a shell script, but after a reboot it was gone. It wasn't reliable and not even a solution at all. I went ahead and removed the code, because the co-mounted cgroups work fine. The real issue was the config line we put in `/etc/libvirt/qemu.conf` so much better to fix that instead.

The default for `libvirtd` is to use only cgroups that are mounted. When you remove the 'cgroup_controllers' line, the default kicks in. According to the `qemu.conf` file, the default is:

`cgroup_controllers = ["cpu", "devices", "memory", "blkio", "cpuacct", "net_cls"]`

Then again, if it's not mounted, it's not used. So, that's a nice way to control it.
This works just fine for both RHEL/CentOS 6 and RHEL/CentOS 7.

Along the debugging, I came across some small issues here and there which I also fixed in this PR.

If you test this, be sure to build RPMs from this branch and install those on the KVM hypervisor.

At Schuberg Philis we run KVM hypervisors on CentOS 7 for months with these fixes. Now I took the time to fix the issues in CloudStack, rather than work around them.

@bhaisaab @karuturi could you please review this? Thanks!

Please note:
The next step is to fix CLOUDSTACK-8625 (Systemd profile for KVM Agent) as currently a sysvinit script is still used. Although it works, we do not have any systemd benefits. As it is a separate issue, I'll try to address it in a separate PR.

* pr/731:
  CLOUDSTACK-8443: detect CentOS 7.x as RHEL 7 alike
  CLOUDSTACK-8443: mention the correct logfile
  CLOUDSTACK-8443: display the right hypervisor type
  CLOUDSTACK-8443: don't try to fix co-mounted cgroups

Signed-off-by: Remi Bergsma <github@remi.nl>
This commit is contained in:
Remi Bergsma 2015-08-25 10:32:27 +02:00
commit c0fde9a12b
6 changed files with 5 additions and 100 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
agent/bindir/cloud-setup-agent.in
View File

@ -65,7 +65,7 @@ def getUserInputs():
if oldHypervisor == "":
oldHypervisor = "kvm"
hypervisor = raw_input("Please input the Hypervisor type kvm/lxc:[%s]"%oldCluster)
hypervisor = raw_input("Please input the Hypervisor type kvm/lxc:[%s]"%oldHypervisor)
if hypervisor == "":
hypervisor = oldHypervisor

23
plugins/hypervisors/kvm/src/com/cloud/hypervisor/kvm/resource/LibvirtComputingResource.java
View File

@ -180,7 +180,6 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
private String _ovsPvlanVmPath;
private String _routerProxyPath;
private String _ovsTunnelPath;
private String _setupCgroupPath;
private String _host;
private String _dcId;
private String _pod;
@ -699,17 +698,6 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
_hypervisorType = HypervisorType.KVM;
}
//Verify that cpu,cpuacct cgroups are not co-mounted
if(HypervisorType.LXC.equals(getHypervisorType())){
_setupCgroupPath = Script.findScript(kvmScriptsDir, "setup-cgroups.sh");
if (_setupCgroupPath == null) {
throw new ConfigurationException("Unable to find the setup-cgroups.sh");
}
if(!checkCgroups()){
throw new ConfigurationException("cpu,cpuacct cgroups are co-mounted");
}
}
_hypervisorURI = (String)params.get("hypervisor.uri");
if (_hypervisorURI == null) {
_hypervisorURI = LibvirtConnection.getHypervisorURI(_hypervisorType.toString());
@ -3332,17 +3320,6 @@ public class LibvirtComputingResource extends ServerResourceBase implements Serv
return _hypervisorType;
}
private boolean checkCgroups(){
final Script command = new Script(_setupCgroupPath, 5 * 1000, s_logger);
String result;
result = command.execute();
if (result != null) {
s_logger.debug("cgroup check failed:" + result);
return false;
}
return true;
}
public String mapRbdDevice(final KVMPhysicalDisk disk){
final KVMStoragePool pool = disk.getPool();
//Check if rbd image is already mapped

19
python/lib/cloud_utils.py
View File

@ -685,24 +685,6 @@ class SetupCgRules(ConfigTask):
enable_service("cgred")
class SetupCgroupControllers(ConfigTask):
name = "qemu cgroup controllers setup"
cfgline = "cgroup_controllers = [ \"cpu\" ]"
filename = "/etc/libvirt/qemu.conf"
def done(self):
try:
return self.cfgline in file(self.filename,"r").read(-1)
except IOError,e:
if e.errno is 2: raise TaskFailed("qemu has not been properly installed on this system")
raise
def execute(self):
libvirtqemu = file(self.filename,"r").read(-1)
libvirtqemu = libvirtqemu + "\n" + self.cfgline + "\n"
file("/etc/libvirt/qemu.conf","w").write(libvirtqemu)
class SetupSecurityDriver(ConfigTask):
name = "security driver setup"
cfgline = "security_driver = \"none\""
@ -887,7 +869,6 @@ def config_tasks(brname, pubNic, prvNic):
SetupNetworking(brname, pubNic, prvNic),
SetupCgConfig(),
SetupCgRules(),
SetupCgroupControllers(),
SetupSecurityDriver(),
SetupLibvirt(),
SetupLiveMigration(),

7
python/lib/cloudutils/serviceConfig.py
View File

@ -56,7 +56,7 @@ class serviceCfgBase(object):
if self.syscfg.env.mode == "Server":
raise CloudRuntimeException("Configure %s failed, Please check the /var/log/cloudstack/setupManagement.log for detail"%self.serviceName)
else:
raise CloudRuntimeException("Configure %s failed, Please check the /var/log/cloudstack/setupAgent.log for detail"%self.serviceName)
raise CloudRuntimeException("Configure %s failed, Please check the /var/log/cloudstack/agent/setup.log for detail"%self.serviceName)
def backup(self):
if self.status is None:
@ -428,7 +428,7 @@ class securityPolicyConfigUbuntu(serviceCfgBase):
return True
except:
raise CloudRuntimeException("Failed to configure apparmor, please see the /var/log/cloudstack/setupAgent.log for detail, \
raise CloudRuntimeException("Failed to configure apparmor, please see the /var/log/cloudstack/agent/setup.log for detail, \
or you can manually disable it before starting myCloud")
def restore(self):
@ -458,7 +458,7 @@ class securityPolicyConfigRedhat(serviceCfgBase):
cfo.replace_line("SELINUX=", "SELINUX=permissive")
return True
except:
raise CloudRuntimeException("Failed to configure selinux, please see the /var/log/cloudstack/setupAgent.log for detail, \
raise CloudRuntimeException("Failed to configure selinux, please see the /var/log/cloudstack/agent/setup.log for detail, \
or you can manually disable it before starting myCloud")
else:
return True
@ -493,7 +493,6 @@ class libvirtConfigRedhat(serviceCfgBase):
filename = "/etc/libvirt/qemu.conf"
cfo = configFileOps(filename, self)
cfo.addEntry("cgroup_controllers", "[\"cpu\"]")
cfo.addEntry("security_driver", "\"none\"")
cfo.addEntry("user", "\"root\"")
cfo.addEntry("group", "\"root\"")

2
python/lib/cloudutils/utilities.py
View File

@ -112,7 +112,7 @@ class Distribution:
version = file("/etc/redhat-release").readline()
if version.find("Red Hat Enterprise Linux Server release 6") != -1 or version.find("Scientific Linux release 6") != -1 or version.find("CentOS Linux release 6") != -1 or version.find("CentOS release 6.") != -1:
self.distro = "RHEL6"
elif version.find("Red Hat Enterprise Linux Server release 7") != -1:
elif version.find("Red Hat Enterprise Linux Server release 7") != -1 or version.find("Scientific Linux release 7") != -1 or version.find("CentOS Linux release 7") != -1 or version.find("CentOS release 7.") != -1:
self.distro = "RHEL7"
elif version.find("CentOS release") != -1:
self.distro = "CentOS"

52
scripts/vm/hypervisor/kvm/setup-cgroups.sh
View File

@ -1,52 +0,0 @@
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# Script to fix cgroups co-mounted issue
# Applies to RHEL 7 versions (and family member CentOS 7) only
# Detect if cpu,cpuacct cgroups are co-mounted
# If co-mounted, unmount and mount them seperately
#set -x
# Check distribution version for RHEL
if [ -f '/etc/redhat-release' ]; then
# Check RHEL version for 7
if grep -E 'Red Hat Enterprise Linux Server release 7|CentOS Linux release 7' /etc/redhat-release > /dev/null; then
# Check if cgroups if co-mounted
if [ -d '/sys/fs/cgroup/cpu,cpuacct' ]; then
# cgroups co-mounted. Requires remount
umount /sys/fs/cgroup/cpu,cpuacct
rm /sys/fs/cgroup/cpu
rm /sys/fs/cgroup/cpuacct
rm -rf /sys/fs/cgroup/cpu,cpuacct
mkdir -p /sys/fs/cgroup/cpu
mkdir -p /sys/fs/cgroup/cpuacct
mount -t cgroup -o cpu cpu "/sys/fs/cgroup/cpu"
mount -t cgroup -o cpuacct cpuacct "/sys/fs/cgroup/cpuacct"
# Verify that cgroups are not co-mounted
if [ -d '/sys/fs/cgroup/cpu,cpuacct' ]; then
echo "cgroups still co-mounted"
exit 1;
fi
fi
fi
fi
exit 0