CLOUDSTACK-10194: Use packer for building systemvmtemplates

- This migrates the current systemvmtemplate build system from
  veewee/virtualbox to packer and qemu based.
- This also introduces and updates a CentOS7 built-in template.
- Remove old appliance build scripts and files.
- Adds iftop package (CLOUDSTACK-9785)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This commit is contained in:
Rohit Yadav 2017-12-17 12:55:26 +05:30
parent 60a216bcc9
commit 3839239a21
61 changed files with 343 additions and 2602 deletions

View File

@ -880,14 +880,8 @@
<exclude>services/secondary-storage/conf/agent.properties</exclude>
<exclude>services/secondary-storage/conf/environment.properties</exclude>
<exclude>test/systemvm/README.md</exclude>
<exclude>tools/appliance/.ruby-version</exclude>
<exclude>tools/vagrant/systemvm/vagrant.pub</exclude>
<exclude>tools/vagrant/systemvm/.ruby-version</exclude>
<exclude>tools/devcloud/basebuild/puppet-devcloudinitial/files/network.conf</exclude>
<exclude>tools/appliance/definitions/devcloud/*</exclude>
<exclude>tools/appliance/definitions/systemvmtemplate/*</exclude>
<exclude>tools/appliance/definitions/systemvm64template/*</exclude>
<exclude>tools/appliance/definitions/builtin/*</exclude>
<exclude>tools/appliance/*/template.json</exclude>
<exclude>tools/cli/cloudmonkey.egg-info/*</exclude>
<exclude>tools/devcloud/src/deps/boxes/basebox-build/definition.rb</exclude>
<exclude>tools/devcloud/src/deps/boxes/basebox-build/preseed.cfg</exclude>

View File

@ -1 +0,0 @@
2.3.0p0

View File

@ -1,24 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
rvm use ruby-2.3.0@vagrant-release-cloudstack --create
export VAGRANT_HOME=$HOME/.vagrant.d-release-cloudstack
bundle check > /dev/null 2>&1
RETVAL=$?
if [ $RETVAL -ne 0 ]; then
bundle install
fi

View File

@ -17,104 +17,27 @@ under the License.
===========================================================
# Introduction
This is used to build appliances for use with CloudStack. Currently two
build profiles are available for building systemvmtemplate (Debian based) and
CentOS based built-in user VM template.
# Setting up Tools and Environment
- Install latest VirtualBox (5.0+)
- Install tools for exporting appliances: qemu-img, vboxmanage, vhd-util, ovftool
- Install [RVM](https://rvm.io/rvm/install)
- Install dependencies (tested on Ubuntu 16.04):
apt-get install sharutils libxslt1-dev libxml2-dev zlib1g-dev build-essential ruby ruby-bundler ruby-dev qemu-utils blktap-utils faketime
- Setup paths:
export PATH=~/.rvm/bin:$PATH
- Install Ruby 2.3.0, if it installed some other version:
rvm install 2.3.0
- Set rvm to use that 2.3.0
rvm use ruby-2.3.0
- Install bundler: (if you get any openssl issue see https://rvm.io/packages/openssl)
gem install bundler
- Install packer and latest KVM, qemu on a Linux machine
- Install tools for exporting appliances: qemu-img, ovftool, faketime
- Build and install `vhd-util` as described in build.sh or use pre-built
binaries at:
All the dependencies will be fetched automatically.
http://packages.shapeblue.com/systemvmtemplate/vhd-util
http://packages.shapeblue.com/systemvmtemplate/libvhd.so.1.0
To save some time if you've downloaded iso of your distro, put the isos in:
tools/appliance/iso/
# Setting up jenkins (CI) builds
All the tools listed above are expected to be available. If you follow
http://rvm.io/integration/jenkins
then you'll need to do a bit of logic to load RVM in jenkins. In the
build script you put into jenkins, start it with
```
#!/bin/bash -l
```
to ensure a login shell, then add something like
```
# inspired by https://github.com/CloudBees-community/rubyci-clickstart/blob/master/bin/run-ci
# also see https://rvm.io/integration/jenkins
# .rvmrc won't get trusted/auto-loaded by jenkins by default
export VAGRANT_HOME=$HOME/.vagrant.d-release-cloudstack
rvm use ruby-1.9.3@vagrant-release-cloudstack --create
# do not use --deployment since that requires Gemfile.lock...and we prefer an up-to-date veewee
bundle_args="--path vendor/bundle"
```
# How to build SystemVMs automatically
# How to build appliances
Just run build.sh, it will export archived appliances for KVM, XenServer,
VMWare and HyperV in `dist` directory:
bundle install
bash build.sh systemvm64template
bash build.sh systemvmtemplate
bash build.sh builtin
# Building SystemVM template appliance manually
List available appliances one can build:
veewee vbox list
Modify scripts in definitions/*appliance*/ as per needs.
Build systemvm template appliance:
veewee vbox build 'systemvmtemplate'
Start the box:
veewee vbox up 'systemvmtemplate'
Halt the box:
veewee vbox halt 'systemvmtemplate'
Now VirtualBox can be used to export appliance.
To build the systemvm64template by hand using veewee, set VM_ARCH=amd64 and use
the systemvmtemplate:
export VM_ARCH=amd64
cp -r definitions/systemvmtemplate definitions/systemvm64template
veewee vbox build 'systemvm64template'
Troubleshooting
===============
If you see following line in the screen, then veewee is failing
extracting vboxmanage version.
Downloading vbox guest additions iso v - http://download.virtualbox.org/vi
You would be able to check it manually by typing:
vboxmanage --version
If you're using Fedora for example, you'll need to install `kernel-devel`
package and run `/etc/init.d/vboxdrv setup` to get veewee working.
Testing
=======
The ./test.sh script tries out a few different default ways to invoke build.sh.
See ../vagrant/systemvm for a test setup that uses vagrant+serverspec to
provide actual integration tests that verify the built systemvm is up to spec.

View File

@ -1,6 +1,4 @@
#!/bin/bash -l
# note: the -l is needed here for bash to always make a login shell and load rvm if it hasn't been loaded
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
@ -18,35 +16,21 @@
# specific language governing permissions and limitations
# under the License.
# build script which wraps around veewee and virtualbox to create the systemvm template
# build script which wraps around packer and virtualbox to create the systemvm template
function usage() {
cat <<END
Usage:
./build.sh [veewee_template [version [branch [BUILD_NUMBER [arch [ssh_key]]]]]
./build.sh [template] [version] [BUILD_NUMBER]
* Set \$appliance to provide veewee definition name to build
* Set \$appliance to provide definition name to build
(or use command line arg, default systemvmtemplate)
* Set \$version to provide version to apply to built appliance
(or use command line arg, default empty)
* Set \$branch to provide branch name to apply to built appliance
(or use command line arg, default from running \`git status\`)
* Set \$BUILD_NUMBER to provide build number to apply to built appliance
(or use command line arg, default empty)
* Set \$arch to provide the (debian) os architecture to inject
(or use command line arg, default i386, other option amd64)
* Set \$ssh_key to provide root ssh public key to inject
(or use command line arg, default set in the veewee definition its authorized_keys.sh)
* Set \$clean_vbox to try pretty hard to remove all our vms and disk from
virtualbox before and after running the rest of the build. This should
not be needed since we try hard to use VBoxManage nicely, but, various
error conditions / timing issues are quite hard to fully contain
* Set \$DEBUG=1 to enable debug logging
* Set \$TRACE=1 to enable trace logging
* Set \$VEEWEE_ARGS to pass veewee custom arguments
(default: empty)
* Set \$VEEWEE_BUILD_ARGS to pass veewee exec build custom arguments
(default: --nogui --auto)
END
exit 0
}
@ -96,51 +80,16 @@ JENKINS_HOME=${JENKINS_HOME:-}
if [[ ! -z "${JENKINS_HOME}" ]]; then
DEBUG=1
fi
VEEWEE_ARGS="${VEEWEE_ARGS:-}"
if [[ "${VEEWEE_ARGS}" == "" && "${TRACE}" == "1" ]]; then
VEEWEE_ARGS="${VEEWEE_ARGS} --debug"
fi
VEEWEE_BUILD_ARGS="${VEEWEE_BUILD_ARGS:-${VEEWEE_ARGS} --nogui --auto}"
# any arguments to pass along to the 'bundle install' command
BUNDLE_ARGS="${BUNDLE_ARGS:-}"
# which veewee definition to use
# which packer definition to use
appliance="${1:-${appliance:-systemvmtemplate}}"
# optional version tag to put into the image filename
version="${2:-${version:-}}"
# branch tag to put into the image filename, populated from `git status` if unset
branch="${3:-${branch:-}}"
# optional (jenkins) build number tag to put into the image filename
BUILD_NUMBER="${4:-${BUILD_NUMBER:-}}"
# (debian) os architecture to build
arch="${5:-${arch:-i386}}"
export VM_ARCH="${arch}"
# optional root SSH public key to write to /root/.ssh/authorized_keys
# note the cs management server overwrites this, so the only reason to
# set this is when working with the VM while it is not under management
# server control
ssh_key="${6:-${ssh_key:-}}"
# whether to attempt to clean up all our virtualbox vms/disks before/after run
clean_vbox="${clean_vbox:-}"
# while building with vbox, we need a quite unique appliance name in order to prevent conflicts with multiple
# concurrent executors on jenkins
if [ -z "${branch}" ] ; then
branch=`(git name-rev --no-undefined --name-only HEAD 2>/dev/null || echo unknown) | sed -e 's/remotes\/.*\///g' | sed -e 's|/|_|g'`
fi
branch_tag=
if [ ! -z "${branch}" ]; then
branch_tag="-${branch}"
fi
version_tag=
if [ ! -z "${version}" ]; then
if [ ! -z "${BUILD_NUMBER}" ]; then
@ -152,10 +101,7 @@ elif [ ! -z "${BUILD_NUMBER}" ]; then
version_tag="-${BUILD_NUMBER}"
fi
appliance_build_name=${appliance}${branch_tag}${version_tag}
# mac address of the vm we create
mac_address=
appliance_build_name=${appliance}${version_tag}
###
### Generic helper functions
@ -244,151 +190,29 @@ function retry() {
###
function create_definition() {
if [ "${appliance}" == "systemvm64template" ]; then
arch="amd64"
export VM_ARCH="${arch}"
rm -rf definitions/systemvm64template # in case of left-over cruft from failed build
cp -r definitions/systemvmtemplate definitions/systemvm64template
add_on_exit rm -rf definitions/systemvm64template
fi
if [ "${appliance}" != "${appliance_build_name}" ]; then
cp -r "definitions/${appliance}" "definitions/${appliance_build_name}"
cp -r "${appliance}" "${appliance_build_name}"
set +e
if [ ! -z "${version}" ]; then
sed ${sed_regex_option} -i -e "s/^CLOUDSTACK_RELEASE=.+/CLOUDSTACK_RELEASE=${version}/" \
"definitions/${appliance_build_name}/configure_systemvm_services.sh"
fi
if [ ! -z "${ssh_key}" ]; then
# ssh key lines can contain /
sed ${sed_regex_option} -i -e "s|^key=.+|key=\"${ssh_key}\"|" \
"definitions/${appliance_build_name}/authorized_keys.sh"
"${appliance_build_name}/configure_systemvm_services.sh"
fi
set -e
add_on_exit rm -rf "definitions/${appliance_build_name}"
add_on_exit rm -rf "${appliance_build_name}"
fi
./shar_cloud_scripts.sh
add_on_exit rm -f cloud_scripts_shar_archive.sh
}
function setup_ruby() {
bundle check || bundle install ${BUNDLE_ARGS}
}
function stop_vbox() {
log INFO "stoppping all virtualbox vms for ${USER}"
bundle exec ./vbox_vm_clean.rb
}
function clean_vbox() {
log INFO "deleting all virtualbox vms and disks for ${USER}"
bundle exec ./vbox_vm_clean.rb --delete --kill
bundle exec ./vbox_disk_clean.rb
}
function prepare() {
log INFO "preparing for build"
setup_ruby
rm -rf dist *.ova *.vhd *.vdi *.qcow* *.bz2 *.vmdk *.ovf
mkdir dist
}
function veewee_destroy() {
log INFO "destroying existing veewee image, if any"
set +e
bundle exec veewee vbox destroy "${appliance_build_name}" ${VEEWEE_ARGS}
set -e
}
function veewee_build() {
log INFO "building new image with veewee"
bundle exec veewee vbox build "${appliance_build_name}" ${VEEWEE_BUILD_ARGS}
}
function veewee_halt() {
log INFO "shutting down new vm with veewee"
bundle exec veewee vbox halt "${appliance_build_name}" ${VEEWEE_ARGS}
}
function save_mac_address() {
log INFO "saving new vm mac address"
mac_address=`vboxmanage showvminfo --details --machinereadable ${appliance_build_name} | grep macaddress1= | sed 's/macaddress1=//' | sed 's/"//g'`
if [ "${mac_address}" == "" ]; then
error "Could not find mac address for appliance ${appliance_build_name}"
fi
}
function check_appliance_shutdown() {
log INFO "waiting for veewee appliance to shut down..."
! (vboxmanage list runningvms | grep "${appliance_build_name}")
local result=$?
if [ ${result} -eq 0 ]; then
log INFO "...veewee appliance shut down ok"
else
log INFO "...veewee appliance still running"
fi
return ${result}
}
function check_appliance_disk_ready() {
log INFO "waiting for veewee appliance disk to be available..."
# local hdd_path="vboxmanage showvminfo '${appliance_build_name}' --machinereadable | \
# egrep '(SATA|IDE) Controller-[0-9]+-[0-9]+' | grep -v '.iso' | \
# grep -v '="none"' | egrep -o '=".*"' | sed 's/=//' | sed 's/"//g'"
local hdd_path=`vboxmanage list hdds | grep "${appliance_build_name}\/" | grep vdi | \
cut -c 14- | sed ${sed_regex_option} 's/^ *//'`
disk_state=`vboxmanage showhdinfo "${hdd_path}" | egrep '^State:' | sed 's/State://' | egrep -o '[a-zA-Z]+' | awk '{print tolower($0)}'`
if [ "${disk_state}" == "notcreated" ]; then
log ERROR "disk ${hdd_path} in state notcreated"
return 1
elif [ "${disk_state}" == "created" ]; then
log INFO "disk ${hdd_path} in state created"
return 0
elif [ "${disk_state}" == "lockedread" ]; then
log INFO "disk ${hdd_path} in state lockedread"
return 1
elif [ "${disk_state}" == "lockedwrite" ]; then
log INFO "disk ${hdd_path} in state lockedwrite"
return 1
elif [ "${disk_state}" == "inaccessible" ]; then
log INFO "disk ${hdd_path} in state inaccessible"
return 1
elif [ "${disk_state}" == "creating" ]; then
log WARN "disk ${hdd_path} in state creating"
return 1
elif [ "${disk_state}" == "deleting" ]; then
log WARN "disk ${hdd_path} in state deleting"
return 1
else
log WARN "disk ${hdd_path} has unknown disk state ${disk_state}"
return 1
fi
}
function remove_shares() {
log INFO "removing shared folders from appliance..."
set +e
local shared_folders=`vboxmanage showvminfo "${appliance_build_name}" | grep Name | grep Host`
if [ "${shared_folders}" == "" ]; then
return 0
fi
folder_name=`echo "${shared_folders}" | head -1 | cut -c 8- | cut -d \' -f 1`
vboxmanage sharedfolder remove "${appliance_build_name}" --name "${folder_name}"
! (vboxmanage showvminfo "${appliance_build_name}" | grep Name | grep Host)
local result=$?
set -e
if [ ${result} -eq 0 ]; then
log INFO "...veewee appliance shared folders removed"
else
log INFO "...veewee appliance still has shared folders"
fi
return ${result}
}
function compact_hdd() {
log INFO "compacting image"
vboxmanage modifyhd "${1}" --compact
function packer_build() {
log INFO "building new image with packer"
cd ${appliance_build_name} && packer build template.json && cd ..
}
function stage_vmx (){
@ -442,13 +266,12 @@ VMXFILE
function xen_server_export() {
log INFO "creating xen server export"
local hdd_path="${1}"
set +e
which faketime >/dev/null 2>&1 && which vhd-util >/dev/null 2>&1
local result=$?
set -e
if [ ${result} == 0 ]; then
vboxmanage internalcommands converttoraw -format vdi "${hdd_path}" img.raw
qemu-img convert -f qcow2 -O raw "dist/${appliance}" img.raw
vhd-util convert -s 0 -t 1 -i img.raw -o stagefixed.vhd
faketime '2010-01-01' vhd-util convert -s 1 -t 2 -i stagefixed.vhd -o "${appliance_build_name}-xen.vhd"
rm -f *.bak
@ -463,117 +286,48 @@ function xen_server_export() {
function ovm_export() {
log INFO "creating OVM export"
local hdd_path="${1}"
rm -f img.raw
vboxmanage internalcommands converttoraw -format vdi "${hdd_path}" img.raw
mv img.raw ${appliance_build_name}-ovm.raw
bzip2 "${appliance_build_name}-ovm.raw"
mv "${appliance_build_name}-ovm.raw.bz2" dist/
qemu-img convert -f qcow2 -O raw "dist/${appliance}" "dist/${appliance_build_name}-ovm.raw"
cd dist && bzip2 "${appliance_build_name}-ovm.raw" && cd ..
log INFO "${appliance} exported for OracleVM: dist/${appliance_build_name}-ovm.raw.bz2"
}
function kvm_export() {
set +e
which faketime >/dev/null 2>&1 && which vhd-util >/dev/null 2>&1
local result=$?
set -e
if [ ${result} == 0 ]; then
log INFO "creating kvm export"
local hdd_path="${1}"
rm -f raw.img
vboxmanage internalcommands converttoraw -format vdi "${hdd_path}" raw.img
set +e
qemu-img convert -o compat=0.10 -f raw -c -O qcow2 raw.img "${appliance_build_name}-kvm.qcow2"
qemu-img convert -o compat=0.10 -f qcow2 -c -O qcow2 "dist/${appliance}" "dist/${appliance_build_name}-kvm.qcow2"
local qemuresult=$?
set -e
if [ ${qemuresult} != 0 ]; then
log INFO "'qemu-img convert' failed, trying without compat option"
qemu-img convert -f raw -c -O qcow2 raw.img "${appliance_build_name}-kvm.qcow2"
fi
add_on_exit rm -f raw.img
bzip2 "${appliance_build_name}-kvm.qcow2"
mv "${appliance_build_name}-kvm.qcow2.bz2" dist/
cd dist && bzip2 "${appliance_build_name}-kvm.qcow2" && cd ..
log INFO "${appliance} exported for KVM: dist/${appliance_build_name}-kvm.qcow2.bz2"
else
log WARN "** Skipping ${appliance_build_name} export for KVM: qemu-img is missing. **"
fi
}
function vmware_export() {
log INFO "creating vmware export"
local machine_uuid="${1}"
local hdd_uuid="${2}"
vboxmanage clonehd "${hdd_uuid}" "${appliance_build_name}-vmware.vmdk" --format VMDK
qemu-img convert -f qcow2 -O vmdk "dist/${appliance}" "dist/${appliance_build_name}-vmware.vmdk"
if ! ovftool_loc="$(type -p "ovftool")" || [ -z "$ovftool_loc" ]; then
log INFO "ovftool not found, using traditional method to export ova file"
vboxmanage export "${machine_uuid}" --output "${appliance_build_name}-vmware.ovf"
log INFO "${appliance} exported for VMWare: dist/${appliance_build_name}-vmware.{vmdk.bz2,ovf}"
add_on_exit rm -f ${appliance_build_name}-vmware.ovf
add_on_exit rm -f ${appliance_build_name}-vmware-disk[0-9].vmdk
log INFO "ovftool not found, skipping ova generation for VMware"
return
fi
# xsltproc doesn't support this XSLT so we use java to run this one XSLT
mv ${appliance_build_name}-vmware.ovf ${appliance_build_name}-vmware.ovf-orig
java -cp convert Convert convert_ovf_vbox_to_esx.xslt \
${appliance_build_name}-vmware.ovf-orig \
${appliance_build_name}-vmware.ovf
add_on_exit rm -f ${appliance_build_name}-vmware.ovf-orig
chmod 666 *.vmdk *.ovf
tar -cf ${appliance_build_name}-vmware.ova \
${appliance_build_name}-vmware.ovf \
${appliance_build_name}-vmware-disk[0-9].vmdk
else
log INFO "ovftool found, using it to export ova file"
CDIR=$PWD
cd dist
chmod 666 ${appliance_build_name}-vmware.vmdk
stage_vmx ${appliance_build_name}-vmware ${appliance_build_name}-vmware.vmdk
ovftool ${appliance_build_name}-vmware.vmx ${appliance_build_name}-vmware.ova
fi
mv ${appliance_build_name}-vmware.ova dist/
rm -f *vmx *vmdk
cd $CDIR
log INFO "${appliance} exported for VMWare: dist/${appliance_build_name}-vmware.ova"
}
function vagrant_export() {
log INFO "creating vagrant export"
local machine_uuid="${1}"
# this is based on veewee export logic, but, we don't want to use veewee export,
# since it makes optimistic assumptions about VM shutdown/halt leading to available
# disks and the like
# predicting VBoxManage disk naming seems problematic
# disk="${appliance_build_name}-vmware.ovf"
# image="${appliance_build_name}-vmware-disk1.vmdk"
mkdir -p "box/${appliance_build_name}"
# cp "${disk}" "box/${appliance_build_name}/box.ovf"
# cp "${image}" "box/${appliance_build_name}/box-disk1.vmdk"
cat >box/${appliance_build_name}/Vagrantfile <<END
Vagrant::Config.run do |config|
# This Vagrantfile is auto-generated by vagrant package to contain
# the MAC address of the box. Custom configuration should be placed in
# the actual Vagrantfile in this box.
config.vm.base_mac = "${mac_address}"
# This is not enough config to be able to use the systemvm .box
# See ../vagrant/systemvm/Vagrantfile for the trickery needed
end
# Load include vagrant file if it exists after the auto-generated
# so it can override any of the settings
include_vagrantfile = File.expand_path("../include/_Vagrantfile", __FILE__)
load include_vagrantfile if File.exist?(include_vagrantfile)
END
vboxmanage export "${machine_uuid}" --output "box/${appliance_build_name}/box.ovf"
( cd box/${appliance_build_name}; tar cf "../${appliance_build_name}.box" . )
mv "box/${appliance_build_name}.box" dist/
log INFO "${appliance} exported for vagrant: dist/${appliance_build_name}.box"
}
function hyperv_export() {
log INFO "creating hyperv export"
local hdd_uuid="${1}"
vboxmanage clonehd "${hdd_uuid}" "${appliance_build_name}-hyperv.vhd" --format VHD
# HyperV doesn't support import a zipped image from S3,
# but we create a zipped version to save space on the jenkins box
qemu-img convert -f qcow2 -O vpc "dist/${appliance}" "dist/${appliance_build_name}-hyperv.vhd"
CDIR=$PWD
cd dist
zip "${appliance_build_name}-hyperv.vhd.zip" "${appliance_build_name}-hyperv.vhd"
add_on_exit rm "${appliance_build_name}-hyperv.vhd"
mv "${appliance_build_name}-hyperv.vhd.zip" dist/
rm -f *vhd
cd $CDIR
log INFO "${appliance} exported for HyperV: dist/${appliance_build_name}-hyperv.vhd.zip"
}
@ -583,37 +337,19 @@ function hyperv_export() {
function main() {
prepare
if [ "${clean_vbox}" == "1" ]; then
clean_vbox
add_on_exit clean_vbox
else
stop_vbox
fi
create_definition
veewee_destroy # in case of left-over cruft from failed build
add_on_exit veewee_destroy
veewee_build
save_mac_address
veewee_halt || true
retry 10 check_appliance_shutdown
retry 10 check_appliance_disk_ready
retry 10 remove_shares
packer_build
# Get appliance uuids
local vm_info=`vboxmanage showvminfo "${appliance_build_name}"`
local machine_uuid=`echo "${vm_info}" | grep UUID | head -1 | awk '{print $2}'`
local hdd_uuid=`echo "${vm_info}" | grep vdi | head -1 | awk '{print $8}' | cut -d ')' -f 1`
local hdd_path=`vboxmanage list hdds | grep "${appliance_build_name}\/" | grep vdi | \
cut -c 14- | sed ${sed_regex_option} 's/^ *//'`
compact_hdd "${hdd_uuid}"
xen_server_export "${hdd_path}"
ovm_export "${hdd_path}"
kvm_export "${hdd_path}"
vmware_export "${machine_uuid}" "${hdd_uuid}"
hyperv_export "${hdd_uuid}"
#vagrant_export "${machine_uuid}"
md5sum dist/* > dist/md5sum.txt
# process the disk at dist
kvm_export
ovm_export
xen_server_export
vmware_export
hyperv_export
rm -f "dist/${appliance}"
cd dist && md5sum * > md5sum.txt && cd ..
cd dist && sha512sum * > sha512sum.txt && cd ..
add_on_exit log INFO "BUILD SUCCESSFUL"
}

View File

@ -0,0 +1,91 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# Run the installer
install
# Use cdrom media
cdrom
lang en_US.UTF-8
keyboard us
network --bootproto=dhcp --hostname=builtin-centos7
rootpw password
firewall --disabled
selinux --disabled
timezone UTC
unsupported_hardware
bootloader --location=mbr
text
skipx
zerombr
clearpart --all --initlabel
autopart
auth --enableshadow --passalgo=sha512 --kickstart
firstboot --disabled
eula --agreed
services --disabled="chronyd" --enabled="sshd"
reboot
user --name=cloud --plaintext --password cloud
%packages --ignoremissing --excludedocs
@^minimal
@core
kexec-tools
# unnecessary firmware
-aic94xx-firmware
-atmel-firmware
-b43-openfwwf
-bfa-firmware
-ipw2100-firmware
-ipw2200-firmware
-ivtv-firmware
-iwl100-firmware
-iwl1000-firmware
-iwl3945-firmware
-iwl4965-firmware
-iwl5000-firmware
-iwl5150-firmware
-iwl6000-firmware
-iwl6000g2a-firmware
-iwl6050-firmware
-libertas-usb8388-firmware
-ql2100-firmware
-ql2200-firmware
-ql23xx-firmware
-ql2400-firmware
-ql2500-firmware
-rt61pci-firmware
-rt73usb-firmware
-xorg-x11-drv-ati-firmware
-zd1211-firmware
%end
%post
exec < /dev/tty3 > /dev/tty3
chvt 3
echo
echo "################################"
echo "# Running Post Configuration #"
echo "################################"
(
echo "cloud ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers
sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers
/usr/bin/yum -y --enablerepo extras install epel-release
) 2>&1 | /usr/bin/tee /var/log/post_install.log
chvt 1
%end

View File

@ -15,6 +15,17 @@
# specific language governing permissions and limitations
# under the License.
source 'https://rubygems.org'
set -eux
gem 'vagrant-wrapper'
sed -i 's/^GSS/\#GSS/g' /etc/ssh/sshd_config
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
echo "UseDNS no" >> /etc/ssh/sshd_config
yum -y update
yum install -y tmux vim htop wget telnet tcpdump sed awk ssh httpd net-tools open-vm-tools
systemctl enable httpd
iptables -I INPUT -p tcp -m tcp --dport 22 -j ACCEPT
iptables -I INPUT -p tcp -m tcp --dport 80 -j ACCEPT
iptables-save > /etc/sysconfig/iptables

View File

@ -15,11 +15,17 @@
# specific language governing permissions and limitations
# under the License.
source 'https://rubygems.org'
set -ux
gem 'veewee', '~> 0.4.5.1'
gem 'em-winrm'
gem 'sys-proctable'
gem 'net-scp'
gem 'net-ssh'
gem 'progressbar', '0.21.0'
rm -f /etc/udev/rules.d/70-persistent*
yum -y clean all
fstrim -av
dd if=/dev/zero of=/EMPTY bs=1M
rm -f /EMPTY
# Block until the empty file has been removed, otherwise, Packer
# will try to kill the box while the disk is still full and that's bad
sync
sync
sync

View File

@ -0,0 +1,53 @@
{
"description": "CentOS 7 x86_64 Built-in template for CloudStack",
"provisioners": [
{
"type": "shell",
"scripts": [
"scripts/base.sh",
"scripts/cleanup.sh"
]
}
],
"builders": [
{
"type": "qemu",
"accelerator": "kvm",
"headless": true,
"qemuargs": [
[ "-m", "512M" ],
[ "-smp", "cpus=1,maxcpus=1,cores=1" ]
],
"disk_size": 10240,
"format": "qcow2",
"disk_interface": "virtio",
"net_device": "virtio-net",
"iso_url": "http://mirror.nbrc.ac.in/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1708.iso",
"iso_checksum": "5848f2fd31c7acf3811ad88eaca6f4aa",
"iso_checksum_type": "md5",
"vm_name": "builtin",
"output_directory": "../dist",
"http_directory": "http",
"http_port_min": 10082,
"http_port_max": 10089,
"ssh_host_port_min": 2222,
"ssh_host_port_max": 2229,
"ssh_username": "root",
"ssh_password": "password",
"ssh_port": 22,
"ssh_wait_timeout": "30m",
"boot_wait": "10s",
"boot_command": [
"<up><wait><tab><wait> text ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}//kickstart.cfg<enter><wait>"
],
"shutdown_command": "shutdown -P now"
}
]
}

View File

@ -1,36 +0,0 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
import javax.xml.transform.TransformerFactory;
import javax.xml.transform.Source;
import javax.xml.transform.Transformer;
import javax.xml.transform.stream.StreamResult;
import javax.xml.transform.stream.StreamSource;
import java.io.File;
import java.io.IOException;
import java.net.URISyntaxException;
public class Convert {
public static void main(String[] args) throws IOException, URISyntaxException, TransformerException {
TransformerFactory factory = TransformerFactory.newInstance();
Source xslt = new StreamSource(new File(args[0]));
Transformer transformer = factory.newTransformer(xslt);
Source text = new StreamSource(new File(args[1]));
transformer.transform(text, new StreamResult(new File(args[2])));
}
}

View File

@ -1,42 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<xsl:stylesheet version="1.0" xmlns="http://schemas.dmtf.org/ovf/envelope/1" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:ovf="http://schemas.dmtf.org/ovf/envelope/1" xmlns:rasd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_ResourceAllocationSettingData" xmlns:vssd="http://schemas.dmtf.org/wbem/wscim/1/cim-schema/2/CIM_VirtualSystemSettingData" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:vbox="http://www.virtualbox.org/ovf/machine">
<xsl:template match="vssd:VirtualSystemType/text()">vmx-07</xsl:template>
<xsl:template match="ovf:Item[./rasd:ResourceType/text()=20]">
<Item>
<xsl:copy-of select="rasd:Address"/>
<rasd:Caption>scsiController0</rasd:Caption>
<rasd:Description>SCSI Controller</rasd:Description>
<rasd:ElementName>scsiController0</rasd:ElementName>
<xsl:copy-of select="rasd:InstanceID"/>
<rasd:ResourceSubType>lsilogic</rasd:ResourceSubType>
<rasd:ResourceType>6</rasd:ResourceType>
</Item>
</xsl:template>
<xsl:template match="node()|@*">
<xsl:copy>
<xsl:apply-templates select="node()|@*"/>
</xsl:copy>
</xsl:template>
</xsl:stylesheet>

View File

@ -1,14 +0,0 @@
# Base install
sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers
cat > /etc/yum.repos.d/epel.repo << EOM
[epel]
name=epel
baseurl=http://download.fedoraproject.org/pub/epel/6/\$basearch
enabled=1
gpgcheck=0
EOM
# Make ssh faster by not waiting on DNS
echo "UseDNS no" >> /etc/ssh/sshd_config

View File

@ -1,21 +0,0 @@
# Clean up unneeded packages.
yum -y erase gtk2 libX11 hicolor-icon-theme avahi freetype bitstream-vera-fonts
yum -y clean all
#rm -rf /etc/yum.repos.d/{puppetlabs,epel}.repo
rm -rf VBoxGuestAdditions_*.iso
rm -rf xs-tools*.iso
# Ensure that udev doesn't screw us with network device naming.
ln -sf /dev/null /lib/udev/rules.d/75-persistent-net-generator.rules
rm -f /etc/udev/rules.d/70-persistent-net.rules
# On startup, remove HWADDR from the eth0 interface.
cp -f /etc/sysconfig/network-scripts/ifcfg-eth0 /tmp/eth0
sed "/^HWADDR/d" /tmp/eth0 > /etc/sysconfig/network-scripts/ifcfg-eth0
sed -e "s/dhcp/none/;s/eth0/eth1/" /etc/sysconfig/network-scripts/ifcfg-eth0 > /etc/sysconfig/network-scripts/ifcfg-eth1
# Prevent way too much CPU usage in VirtualBox by disabling APIC.
sed -e 's/\tkernel.*/& noapic/' /boot/grub/grub.conf > /tmp/new_grub.conf
mv /boot/grub/grub.conf /boot/grub/grub.conf.bak
mv /tmp/new_grub.conf /boot/grub/grub.conf

View File

@ -1,33 +0,0 @@
Veewee::Definition.declare({
:cpu_count => '1',
:memory_size=> '2048',
:disk_size => '8000', :disk_format => 'VDI', :hostiocache => 'off',
:os_type_id => 'RedHat6_64',
:iso_file => "CentOS-6.4-x86_64-minimal.iso",
:iso_src => "http://centos.mirror.net.in/centos/6.4/isos/x86_64/CentOS-6.4-x86_64-minimal.iso",
:iso_md5 => "4a5fa01c81cc300f4729136e28ebe600",
:iso_download_timeout => "1000",
:boot_wait => "10",
:boot_cmd_sequence => [
'<Tab> text ks=http://%IP%:%PORT%/ks.cfg<Enter>'
],
:kickstart_port => "7122",
:kickstart_timeout => "10000",
:kickstart_file => "ks.cfg",
:ssh_login_timeout => "10000",
:ssh_user => "root",
:ssh_password => "password",
:ssh_key => "",
:ssh_host_port => "7222",
:ssh_guest_port => "22",
:sudo_cmd => "echo '%p'|sudo -S sh '%f'",
:shutdown_cmd => "halt -p",
:postinstall_files => [
"base.sh",
"postinstall.sh",
"install-xs-tools.sh",
"cleanup.sh",
"zerodisk.sh"
],
:postinstall_timeout => "10000"
})

View File

@ -1,10 +0,0 @@
# get the latest xs tools available from xen.org
wget --no-check-certificate http://downloads.xen.org/XCP/debian/xs-tools-5.9.960.iso -O xs-tools.iso
sudo mount -o loop xs-tools.iso /mnt
#install the xs tools
sudo yes | sudo sh /mnt/Linux/install.sh
#unmount and cleanup
sudo umount /mnt

View File

@ -1,35 +0,0 @@
install
cdrom
lang en_US.UTF-8
keyboard us
network --bootproto=dhcp
rootpw password
firewall --enabled --service=ssh
authconfig --enableshadow --passalgo=sha512
selinux --disabled
timezone UTC
bootloader --location=mbr
text
skipx
zerombr
clearpart --all --initlabel
autopart
auth --useshadow --enablemd5
firstboot --disabled
reboot
%packages --nobase
@core
%end
%post
/usr/bin/yum -y install sudo
/usr/sbin/groupadd veewee
/usr/sbin/useradd veewee -g veewee -G wheel
echo "veewee"|passwd --stdin veewee
echo "veewee ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/veewee
chmod 0440 /etc/sudoers.d/veewee
%end

View File

@ -1,54 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
set -x
install_packages() {
# dev tools, ssh, nfs
yum -y install wget telnet tcpdump sed awk ssh htop
# utlities
yum -y install httpd
}
start_services() {
service httpd start
}
httpd_configure() {
# start httpd on boot
chkconfig httpd on
# open port 80
iptables -I INPUT -p tcp --dport 80 -j ACCEPT
# create a test page
echo "<h1> Hello, World </h1>" > /var/www/html/test.html
# give 755 permissions and ownership
chmod -R 755 /var/www/html/
chown -R apache:apache /var/www/html/
}
begin=$(date +%s)
install_packages
httpd_configure
start_services
fin=$(date +%s)
t=$((fin-begin))
echo "Testing Builtin baked in $t seconds"

View File

@ -1,3 +0,0 @@
# Zero out the free space to save space in the final image:
dd if=/dev/zero of=/EMPTY bs=1M
rm -f /EMPTY

View File

@ -1,49 +0,0 @@
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
set -e
set -x
function cleanup_apt() {
#apt-get -y remove linux-headers-$(uname -r) build-essential
apt-get -y remove dictionaries-common busybox
apt-get -y autoremove
apt-get autoclean
apt-get clean
}
# Removing leftover leases and persistent rules
function cleanup_dhcp() {
rm -f /var/lib/dhcp/*
}
# Make sure Udev doesn't block our network
function cleanup_dev() {
echo "cleaning up udev rules"
rm -f /etc/udev/rules.d/70-persistent-net.rules
rm -rf /dev/.udev/
rm -f /lib/udev/rules.d/75-persistent-net-generator.rules
}
function cleanup() {
cleanup_apt
cleanup_dhcp
cleanup_dev
}
return 2>/dev/null || cleanup

View File

@ -1,78 +0,0 @@
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
set -e
set -x
function add_admin_group() {
groupadd -f -r admin
}
function configure_cloud_user() {
usermod -a -G admin cloud
mkdir -p /home/cloud/.ssh
chmod 700 /home/cloud/.ssh
echo "cloud:`openssl rand -base64 32`" | chpasswd
}
function configure_sudoers() {
cat >/etc/sudoers <<END
Defaults env_reset
Defaults exempt_group=admin
Defaults mail_badpass
Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
root ALL=(ALL:ALL) ALL
%admin ALL=NOPASSWD:/bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, /bin/umount
#includedir /etc/sudoers.d
END
echo 'cloud ALL=NOPASSWD:/bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, /bin/umount' > /etc/sudoers.d/cloud
}
# sshd_config is overwritten from cloud_scripts
#function configure_sshd() {
# grep "UseDNS no" /etc/ssh/sshd_config && \
# grep "PasswordAuthentication no" /etc/ssh/sshd_config && \
# return
# # Tweak sshd to prevent DNS resolution (speed up logins)
# echo 'UseDNS no' >> /etc/ssh/sshd_config
#
# # Require ssh keys for login
# sed -i -e 's/^.*PasswordAuthentication .*$/PasswordAuthentication no/g' /etc/ssh/sshd_config
#}
function configure_inittab() {
grep "vc:2345:respawn:/sbin/getty" /etc/inittab && return
# Fix inittab
cat >> /etc/inittab << EOF
vc:2345:respawn:/sbin/getty 38400 hvc0
EOF
}
function configure_login() {
add_admin_group
configure_cloud_user
configure_sudoers
# configure_sshd
configure_inittab
}
return 2>/dev/null || configure_login

View File

@ -1,80 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
arch = ENV['VM_ARCH'] || 'i386'
architectures = {
:i386 => {
:os_type_id => 'Debian',
:iso_file => 'debian-7.6.0-i386-netinst.iso',
:iso_src => 'http://cdimage.debian.org/debian-cd/7.6.0/i386/iso-cd/debian-7.6.0-i386-netinst.iso',
:iso_md5 => '528e1a7315da1bbf50bd4d187880a519',
},
:amd64 => {
:os_type_id => 'Debian_64',
:iso_file => 'debian-7.6.0-amd64-netinst.iso',
:iso_src => 'http://cdimage.debian.org/debian-cd/7.6.0/amd64/iso-cd/debian-7.6.0-amd64-netinst.iso',
:iso_md5 => '8a3c2ad7fd7a9c4c7e9bcb5cae38c135'
}
}
config = {
:cpu_count => '1',
:memory_size => '256',
:disk_size => '2500', :disk_format => 'VDI', :hostiocache => 'off',
:iso_download_timeout => '1200',
:boot_wait => '10',
:boot_cmd_sequence => [
'<Esc>',
'install ',
'preseed/url=http://%IP%:%PORT%/preseed.cfg ',
'debian-installer=en_US ',
'auto ',
'locale=en_US ',
'kbd-chooser/method=us ',
'netcfg/get_hostname=systemvm ',
'netcfg/get_domain=apache.org ',
'fb=false ',
'debconf/frontend=noninteractive ',
'console-setup/ask_detect=false ',
'console-keymaps-at/keymap=us ',
'keyboard-configuration/xkb-keymap=us ',
'<Enter>'
],
:kickstart_port => '7122',
:kickstart_timeout => '1200',
:kickstart_file => 'preseed.cfg',
:ssh_login_timeout => '1200',
:ssh_user => 'root',
:ssh_password => 'password',
:ssh_key => '',
:ssh_host_port => '7222',
:ssh_guest_port => '22',
:sudo_cmd => "echo '%p'|sudo -S sh '%f'",
:shutdown_cmd => 'halt -p',
:postinstall_files => [
# basic minimal vm creation
'configure_login.sh',
'cleanup.sh',
'zerodisk.sh'
],
:postinstall_timeout => '1200'
}
config.merge! architectures[arch.to_sym]
Veewee::Definition.declare(config)

View File

@ -1,129 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
### Localization
# Locale sets language and country.
d-i debian-installer/locale string en_US
# Keyboard selection.
d-i console-keymaps-at/keymap select us
### Network configuration
d-i netcfg/choose_interface select auto
d-i netcfg/get_hostname string debianbase
d-i netcfg/get_domain string cloudstack.org
### Mirror settings
d-i mirror/country string manual
d-i mirror/http/hostname string http.us.debian.org
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string
### Clock and time zone setup
d-i clock-setup/utc boolean true
d-i time/zone string UTC
d-i clock-setup/ntp boolean true
### Partitioning
d-i partman-auto/disk string /dev/sda
d-i partman-auto/method string regular
d-i partman-auto/choose_recipe select atomic
d-i partman-auto/expert_recipe string \
boot-root :: \
30 50 100 ext4 \
$primary{ } $bootable{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /boot } \
. \
300 40 400 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ / } \
. \
50 100 200 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /home } \
. \
650 20 1100 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /usr } \
. \
400 40 500 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /opt } \
. \
450 60 1000 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /var } \
. \
50 70 400 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /tmp } \
. \
70 512 300% linux-swap \
method{ swap } format{ } \
.
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
### Base system installation
# ...
### Account setup
d-i passwd/root-login boolean true
d-i passwd/root-password password password
d-i passwd/root-password-again password password
d-i passwd/user-fullname string Cloud Stack
d-i passwd/username string cloud
d-i passwd/user-password password cloud
d-i passwd/user-password-again password cloud
d-i user-setup/encrypt-home boolean false
d-i user-setup/allow-password-weak boolean true
d-i passwd/user-default-groups string audio cdrom video admin
### Apt setup
# ...
### Package selection
tasksel tasksel/first multiselect ssh-server
d-i pkgsel/include string openssh-server ntp acpid sudo bzip2
# Allowed values: none, safe-upgrade, full-upgrade
d-i pkgsel/upgrade select none
popularity-contest popularity-contest/participate boolean false
### Boot loader installation
d-i grub-installer/only_debian boolean true
d-i finish-install/reboot_in_progress note
### Preseeding other packages
libssl1.0.0 libssl1.0.0/restart-services string
libssl1.0.0 libssl1.0.0/restart-failed error
#### Advanced options
# Prevent packaged version of VirtualBox Guest Additions being installed:
d-i preseed/early_command string sed -i \
'/in-target/idiscover(){/sbin/discover|grep -v VirtualBox;}' \
/usr/lib/pre-pkgsel.d/20install-hwpackages

View File

@ -1,43 +0,0 @@
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
set -e
set -x
# clean up stuff copied in by veewee
function cleanup_veewee() {
# this has to be here since it is the last file to run (and we remove ourselves)
rm -fv /root/*.iso
rm -fv /root/{apt_upgrade,authorized_keys,build_time,cleanup,install_systemvm_packages,zerodisk}.sh
rm -fv /root/configure_{acpid,conntrack,grub,locale,login,networking,systemvm_services}.sh
rm -fv .veewee_version .veewee_params .vbox_version
}
# Zero out the free space to save space in the final image:
function zero_disk() {
cleanup_veewee
for path in / /boot /usr /var /opt /tmp /home
do
dd if=/dev/zero of=${path}/zero bs=1M || true
sync
rm -f ${path}/zero
done
}
return 2>/dev/null || zero_disk

View File

@ -1,12 +0,0 @@
# Update the box
apt-get -y update
#below are needed for ruby perhaps
#apt-get -y install linux-headers-$(uname -r) build-essential
#apt-get -y install zlib1g-dev libssl-dev libreadline-gplv2-dev
apt-get -y install curl unzip
apt-get clean
echo 'cloud ALL=NOPASSWD:/bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, /bin/umount' > /etc/sudoers.d/cloud
# Tweak sshd to prevent DNS resolution (speed up logins)
echo 'UseDNS no' >> /etc/ssh/sshd_config

View File

@ -1,21 +0,0 @@
# Clean up
#apt-get -y remove linux-headers-$(uname -r) build-essential
apt-get -y remove dictionaries-common busybox
apt-get -y autoremove
apt-get autoclean
apt-get clean
# Removing leftover leases and persistent rules
echo "cleaning up dhcp leases"
rm /var/lib/dhcp/*
# Make sure Udev doesn't block our network
echo "cleaning up udev rules"
rm /etc/udev/rules.d/70-persistent-net.rules
mkdir /etc/udev/rules.d/70-persistent-net.rules
rm -rf /dev/.udev/
rm /lib/udev/rules.d/75-persistent-net-generator.rules
echo "Adding a 2 sec delay to the interface up, to make the dhclient happy"
echo "pre-up sleep 2" >> /etc/network/interfaces

View File

@ -1,45 +0,0 @@
Veewee::Definition.declare({
:cpu_count => '1',
:memory_size=> '2048',
:disk_size => '32000', :disk_format => 'VDI', :hostiocache => 'off',
:os_type_id => 'Debian',
:iso_file => "debian-7.0.0-i386-netinst.iso",
:iso_src => "http://cdimage.debian.org/mirror/cdimage/archive/7.0.0/i386/iso-cd/debian-7.0.0-i386-netinst.iso",
:iso_md5 => "a6b93666a5393334accb7ac4ee28d949",
:iso_download_timeout => "1000",
:boot_wait => "10", :boot_cmd_sequence => [
'<Esc>',
'install ',
'preseed/url=http://%IP%:%PORT%/preseed.cfg ',
'debian-installer=en_US ',
'auto ',
'locale=en_US ',
'kbd-chooser/method=us ',
'netcfg/get_hostname=systemvm ',
'netcfg/get_domain=apache.org ',
'fb=false ',
'debconf/frontend=noninteractive ',
'console-setup/ask_detect=false ',
'console-keymaps-at/keymap=us ',
'keyboard-configuration/xkb-keymap=us ',
'<Enter>'
],
:kickstart_port => "7122",
:kickstart_timeout => "10000",
:kickstart_file => "preseed.cfg",
:ssh_login_timeout => "10000",
:ssh_user => "root",
:ssh_password => "password",
:ssh_key => "",
:ssh_host_port => "7222",
:ssh_guest_port => "22",
:sudo_cmd => "echo '%p'|sudo -S sh '%f'",
:shutdown_cmd => "halt -p",
:postinstall_files => [
"base.sh",
"postinstall.sh",
"cleanup.sh",
"zerodisk.sh"
],
:postinstall_timeout => "10000"
})

View File

@ -1,60 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
set -x
install_packages() {
DEBIAN_FRONTEND=noninteractive
DEBIAN_PRIORITY=critical
# utlities
apt-get --no-install-recommends -q -y --force-yes install python bzip2 sed gawk diffutils grep gzip less tar telnet wget zip unzip sudo
# dev tools, ssh, nfs
apt-get --no-install-recommends -q -y --force-yes install git vim tcpdump ebtables iptables openssl openssh-server openjdk-6-jdk genisoimage python-pip nfs-kernel-server
# mysql with root password=password
debconf-set-selections <<< 'mysql-server-<version> mysql-server/root_password password password'
debconf-set-selections <<< 'mysql-server-<version> mysql-server/root_password_again password password'
apt-get --no-install-recommends -q -y --force-yes install mysql-server
# xen and xcp
apt-get --no-install-recommends -q -y --force-yes install linux-headers-3.2.0-4-686-pae xen-hypervisor-4.1-i386 xcp-xapi xcp-xe xcp-guest-templates xcp-vncterm xen-tools blktap-utils blktap-dkms qemu-keymaps qemu-utils
}
fix_locale() {
cat >> /etc/default/locale << EOF
LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8
EOF
cat >> /etc/locale.gen << EOF
en_US.UTF-8 UTF-8
EOF
locale-gen en_US.UTF-8
}
begin=$(date +%s)
install_packages
fix_locale
fin=$(date +%s)
t=$((fin-begin))
echo "DevCloud baked in $t seconds"

View File

@ -1,357 +0,0 @@
#### Contents of the preconfiguration file (for squeeze)
### Localization
# Locale sets language and country.
d-i debian-installer/locale string en_US
# Keyboard selection.
#d-i console-tools/archs select at
d-i console-keymaps-at/keymap select us
# Example for a different keyboard architecture
#d-i console-keymaps-usb/keymap select mac-usb-us
### Network configuration
# netcfg will choose an interface that has link if possible. This makes it
# skip displaying a list if there is more than one interface.
d-i netcfg/choose_interface select auto
# To pick a particular interface instead:
#d-i netcfg/choose_interface select eth1
# If you have a slow dhcp server and the installer times out waiting for
# it, this might be useful.
#d-i netcfg/dhcp_timeout string 60
# If you prefer to configure the network manually, uncomment this line and
# the static network configuration below.
#d-i netcfg/disable_dhcp boolean true
# If you want the preconfiguration file to work on systems both with and
# without a dhcp server, uncomment these lines and the static network
# configuration below.
#d-i netcfg/dhcp_failed note
#d-i netcfg/dhcp_options select Configure network manually
# Static network configuration.
#d-i netcfg/get_nameservers string 192.168.1.1
#d-i netcfg/get_ipaddress string 192.168.1.42
#d-i netcfg/get_netmask string 255.255.255.0
#d-i netcfg/get_gateway string 192.168.1.1
#d-i netcfg/confirm_static boolean true
# Any hostname and domain names assigned from dhcp take precedence over
# values set here. However, setting the values still prevents the questions
# from being shown, even if values come from dhcp.
d-i netcfg/get_hostname string systemvm
d-i netcfg/get_domain string cloudstack.org
# Disable that annoying WEP key dialog.
d-i netcfg/wireless_wep string
# The wacky dhcp hostname that some ISPs use as a password of sorts.
#d-i netcfg/dhcp_hostname string radish
# If non-free firmware is needed for the network or other hardware, you can
# configure the installer to always try to load it, without prompting. Or
# change to false to disable asking.
#d-i hw-detect/load_firmware boolean true
### Network console
# Use the following settings if you wish to make use of the network-console
# component for remote installation over SSH. This only makes sense if you
# intend to perform the remainder of the installation manually.
#d-i anna/choose_modules string network-console
#d-i network-console/password password r00tme
#d-i network-console/password-again password r00tme
### Mirror settings
# If you select ftp, the mirror/country string does not need to be set.
#d-i mirror/protocol string ftp
d-i mirror/country string manual
d-i mirror/http/hostname string http.us.debian.org
d-i mirror/http/directory string /debian
d-i mirror/http/proxy string
# Suite to install.
#d-i mirror/suite string testing
# Suite to use for loading installer components (optional).
#d-i mirror/udeb/suite string testing
### Clock and time zone setup
# Controls whether or not the hardware clock is set to UTC.
d-i clock-setup/utc boolean true
# You may set this to any valid setting for $TZ; see the contents of
# /usr/share/zoneinfo/ for valid values.
d-i time/zone string UTC
# Controls whether to use NTP to set the clock during the install
d-i clock-setup/ntp boolean true
# NTP server to use. The default is almost always fine here.
#d-i clock-setup/ntp-server string ntp.example.com
### Partitioning
# If the system has free space you can choose to only partition that space.
#d-i partman-auto/init_automatically_partition select biggest_free
# Alternatively, you can specify a disk to partition. The device name must
# be given in traditional non-devfs format.
# Note: A disk must be specified, unless the system has only one disk.
# For example, to use the first SCSI/SATA hard disk:
d-i partman-auto/disk string /dev/sda
# In addition, you'll need to specify the method to use.
# The presently available methods are: "regular", "lvm" and "crypto"
d-i partman-auto/method string regular
# If one of the disks that are going to be automatically partitioned
# contains an old LVM configuration, the user will normally receive a
# warning. This can be preseeded away...
#d-i partman-lvm/device_remove_lvm boolean true
# The same applies to pre-existing software RAID array:
#d-i partman-md/device_remove_md boolean true
# And the same goes for the confirmation to write the lvm partitions.
#d-i partman-lvm/confirm boolean true
#d-i partman-lvm/confirm_nooverwrite boolean true
#d-i partman/choose_partition select finish
#d-i partman-auto-lvm/guided_size string max
# You can choose one of the three predefined partitioning recipes:
# - atomic: all files in one partition
# - home: separate /home partition
# - multi: separate /home, /usr, /var, and /tmp partitions
d-i partman-auto/choose_recipe select atomic
#d-i partman/default_filesystem string ext3
# Or provide a recipe of your own...
# The recipe format is documented in the file devel/partman-auto-recipe.txt.
# If you have a way to get a recipe file into the d-i environment, you can
# just point at it.
#d-i partman-auto/expert_recipe_file string /hd-media/recipe
d-i partman-auto/expert_recipe string \
boot-root :: \
40 50 100 ext4 \
$primary{ } $bootable{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /boot } \
. \
400 40 500 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ / } \
. \
60 100 200 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /home } \
. \
500 30 1000 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /usr } \
. \
400 40 500 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /opt } \
. \
500 60 1000 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /var } \
. \
100 70 400 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /tmp } \
. \
64 512 300% linux-swap \
method{ swap } format{ } \
.
# If not, you can put an entire recipe into the preconfiguration file in one
# (logical) line. This example creates a small /boot partition, suitable
# swap, and uses the rest of the space for the root partition:
#d-i partman-auto/expert_recipe string \
# boot-root :: \
# 40 50 100 ext3 \
# $primary{ } $bootable{ } \
# method{ format } format{ } \
# use_filesystem{ } filesystem{ ext3 } \
# mountpoint{ /boot } \
# . \
# 500 10000 1000000000 ext3 \
# method{ format } format{ } \
# use_filesystem{ } filesystem{ ext3 } \
# mountpoint{ / } \
# . \
# 64 512 300% linux-swap \
# method{ swap } format{ } \
# .
#The preseed line that "selects finish" needs to be in a certain order in your preseed, the example-preseed does not follow this.
#http://ubuntuforums.org/archive/index.php/t-1504045.html
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
#d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
### Base system installation
# Select the initramfs generator used to generate the initrd for 2.6 kernels.
#d-i base-installer/kernel/linux/initramfs-generators string yaird
# The kernel image (meta) package to be installed; "none" can be used if no
# kernel is to be installed.
#d-i base-installer/kernel/image string linux-image-2.6-486
### Account setup
# Skip creation of a root account (normal user account will be able to
# use sudo).
d-i passwd/root-login boolean true
# Alternatively, to skip creation of a normal user account.
#d-i passwd/make-user boolean false
# Root password, either in clear text
d-i passwd/root-password password password
d-i passwd/root-password-again password password
# or encrypted using an MD5 hash.
#d-i passwd/root-password-crypted password [MD5 hash]
# To create a normal user account.
d-i passwd/user-fullname string Cloud Stack
d-i passwd/username string cloud
# Normal user's password, either in clear text
d-i passwd/user-password password cloud
d-i passwd/user-password-again password cloud
# or encrypted using an MD5 hash.
#d-i passwd/user-password-crypted password [MD5 hash]
# Create the first user with the specified UID instead of the default.
#d-i passwd/user-uid string 1010
d-i user-setup/encrypt-home boolean false
d-i user-setup/allow-password-weak boolean true
# The user account will be added to some standard initial groups. To
# override that, use this.
d-i passwd/user-default-groups string audio cdrom video admin
### Apt setup
# You can choose to install non-free and contrib software.
#d-i apt-setup/non-free boolean true
#d-i apt-setup/contrib boolean true
# Uncomment this if you don't want to use a network mirror.
#d-i apt-setup/use_mirror boolean false
# Select which update services to use; define the mirrors to be used.
# Values shown below are the normal defaults.
#d-i apt-setup/services-select multiselect security, volatile
#d-i apt-setup/security_host string security.debian.org
#d-i apt-setup/volatile_host string volatile.debian.org
# By default the installer requires that repositories be authenticated
# using a known gpg key. This setting can be used to disable that
# authentication. Warning: Insecure, not recommended.
#d-i debian-installer/allow_unauthenticated string true
### Package selection
tasksel tasksel/first multiselect ssh-server
# If the desktop task is selected, install the kde and xfce desktops
# instead of the default gnome desktop.
#tasksel tasksel/desktop multiselect kde, xfce
# Individual additional packages to install
d-i pkgsel/include string openssh-server ntp acpid sudo bzip2
# Whether to upgrade packages after debootstrap.
# Allowed values: none, safe-upgrade, full-upgrade
d-i pkgsel/upgrade select none
# Some versions of the installer can report back on what software you have
# installed, and what software you use. The default is not to report back,
# but sending reports helps the project determine what software is most
# popular and include it on CDs.
popularity-contest popularity-contest/participate boolean false
### Boot loader installation
# Grub is the default boot loader (for x86). If you want lilo installed
# instead, uncomment this:
#d-i grub-installer/skip boolean true
# To also skip installing lilo, and install no bootloader, uncomment this
# too:
#d-i lilo-installer/skip boolean true
# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true
# This one makes grub-installer install to the MBR if it also finds some other
# OS, which is less safe as it might not be able to boot that other OS.
#d-i grub-installer/with_other_os boolean true
# Alternatively, if you want to install to a location other than the mbr,
# uncomment and edit these lines:
#d-i grub-installer/only_debian boolean false
#d-i grub-installer/with_other_os boolean false
#d-i grub-installer/bootdev string (hd0,0)
# To install grub to multiple disks:
#d-i grub-installer/bootdev string (hd0,0) (hd1,0) (hd2,0)
# Optional password for grub, either in clear text
#d-i grub-installer/password password r00tme
#d-i grub-installer/password-again password r00tme
# or encrypted using an MD5 hash, see grub-md5-crypt(8).
#d-i grub-installer/password-crypted password [MD5 hash]
### Finishing up the installation
# During installations from serial console, the regular virtual consoles
# (VT1-VT6) are normally disabled in /etc/inittab. Uncomment the next
# line to prevent this.
#d-i finish-install/keep-consoles boolean true
# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note
# This will prevent the installer from ejecting the CD during the reboot,
# which is useful in some situations.
#d-i cdrom-detect/eject boolean false
# This is how to make the installer shutdown when finished, but not
# reboot into the installed system.
#d-i debian-installer/exit/halt boolean true
# This will power off the machine instead of just halting it.
#d-i debian-installer/exit/poweroff boolean true
### Preseeding other packages
# Depending on what software you choose to install, or if things go wrong
# during the installation process, it's possible that other questions may
# be asked. You can preseed those too, of course. To get a list of every
# possible question that could be asked during an install, do an
# installation, and then run these commands:
# debconf-get-selections --installer > file
# debconf-get-selections >> file
#### Advanced options
### Running custom commands during the installation
# d-i preseeding is inherently not secure. Nothing in the installer checks
# for attempts at buffer overflows or other exploits of the values of a
# preconfiguration file like this one. Only use preconfiguration files from
# trusted locations! To drive that home, and because it's generally useful,
# here's a way to run any shell command you'd like inside the installer,
# automatically.
# This first command is run as early as possible, just after
# preseeding is read.
# Prevent packaged version of VirtualBox Guest Additions being installed:
d-i preseed/early_command string sed -i \
'/in-target/idiscover(){/sbin/discover|grep -v VirtualBox;}' \
/usr/lib/pre-pkgsel.d/20install-hwpackages
# This command is run just before the install finishes, but when there is
# still a usable /target directory. You can chroot to /target and use it
# directly, or use the apt-install and in-target commands to easily install
# packages and run commands in the target system.

View File

@ -1,11 +0,0 @@
# Clean up stuff copied in by veewee
rm -fv /root/*.iso
rm -fv /root/base.sh /root/cleanup.sh /root/postinstall.sh /root/zerodisk.sh
rm -fv .veewee_version .veewee_params .vbox_version
echo "Cleaning up"
# Zero out the free space to save space in the final image:
dd if=/dev/zero of=/zero bs=1M
sync
rm -fv /zero

View File

@ -16,7 +16,7 @@
# specific language governing permissions and limitations
# under the License.
# since veewee wants .sh files to execute, we'll give it a shar
# since packer wants .sh files to execute, we'll give it a shar
set -e
set -x

View File

@ -52,26 +52,25 @@ d-i time/zone string UTC
d-i clock-setup/ntp boolean true
### Partitioning
d-i partman-auto/disk string /dev/sda
d-i partman-auto/disk string /dev/vda
d-i partman-auto/method string regular
d-i partman-auto/choose_recipe select atomic
d-i partman-auto/expert_recipe string \
boot-root :: \
50 50 100 ext2 \
80 50 100 ext2 \
$primary{ } $bootable{ } \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext2 } \
mountpoint{ /boot } \
. \
1300 40 1600 ext4 \
1100 40 1600 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ / } \
. \
350 60 500 ext4 \
600 60 800 ext4 \
method{ format } format{ } \
use_filesystem{ } filesystem{ ext4 } \
mountpoint{ /var/log } \
mountpoint{ /var } \
. \
100 90 200 ext4 \
method{ format } format{ } \
@ -103,7 +102,7 @@ d-i user-setup/encrypt-home boolean false
d-i user-setup/allow-password-weak boolean true
d-i passwd/user-default-groups string audio cdrom video admin
#openssh-server openssh-server/permit-root-login boolean true
openssh-server openssh-server/permit-root-login boolean true
### Apt setup
# ...

View File

@ -49,10 +49,11 @@ function apt_upgrade() {
add_backports
rm -fv /root/*.iso
apt-get -y autoremove
apt-get autoclean
apt-get -q -y update
apt-get -q -y upgrade
apt-get -y autoremove --purge
apt-get autoclean
apt-get clean
}
return 2>/dev/null || apt_upgrade

View File

@ -20,6 +20,10 @@ set -e
set -x
function configure_grub() {
echo "blacklist floppy" > /etc/modprobe.d/blacklist-floppy.conf
rmmod floppy || true
update-initramfs -u
cat > /etc/default/grub <<EOF
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.

View File

@ -22,4 +22,3 @@ set -x
# Create the directory where we store persistent config blobs
mkdir -p /etc/cloudstack
touch /etc/cloudstack/.keep

View File

@ -29,7 +29,7 @@ architectures = {
:os_type_id => 'Debian_64',
:iso_file => 'debian-9.3.0-amd64-netinst.iso',
:iso_src => 'https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.3.0-amd64-netinst.iso',
:iso_sha512 => '8775231d6f56a3d8f116eb64fe048f5cbd2ea0f8c092a1cb7608bcb4106f9c85cb69ce68f53bd381019ab40f1c0316843036daf3fd9107c81c58a240334cc747'
:iso_md5 => '8775231d6f56a3d8f116eb64fe048f5cbd2ea0f8c092a1cb7608bcb4106f9c85cb69ce68f53bd381019ab40f1c0316843036daf3fd9107c81c58a240334cc747'
}
}
@ -77,7 +77,7 @@ config = {
# turning it into a systemvm
'install_systemvm_packages.sh',
'configure_conntrack.sh',
'../../cloud_scripts_shar_archive.sh',
#'../../cloud_scripts_shar_archive.sh',
'configure_systemvm_services.sh',
'authorized_keys.sh',
'configure_persistent_config.sh',

View File

@ -39,8 +39,8 @@ END
echo 'cloud ALL=NOPASSWD:/bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, /bin/umount, /sbin/halt' > /etc/sudoers.d/cloud
}
# clean up stuff copied in by veewee
function cleanup_veewee() {
# clean up stuff copied in
function cleanup_final() {
# this has to be here since it is the last file to run (and we remove ourselves)
cd /home/cloud
rm -fv *.iso
@ -64,10 +64,10 @@ function zero_disk() {
function finalize() {
configure_misc
configure_sudoers
cleanup_veewee
cleanup_final
sync
zero_disk
halt -p
sync
}
return 2>/dev/null || finalize

View File

@ -45,16 +45,9 @@ function install_packages() {
local apt_get="apt-get --no-install-recommends -q -y"
#32 bit architecture support:: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install links:i386 libuuid1:i386 libc6:i386
fi
${apt_get} install grub-legacy \
rsyslog logrotate cron net-tools ifupdown tmux vim htop netbase iptables \
openssh-server e2fsprogs tcpdump socat wget \
openssh-server e2fsprogs tcpdump iftop socat wget \
python bzip2 sed gawk diffutils grep gzip less tar telnet ftp rsync traceroute psmisc lsof procps \
inetutils-ping iputils-arping httping curl \
dnsutils zip unzip ethtool uuid file iproute acpid sudo \
@ -83,6 +76,13 @@ function install_packages() {
apt-get autoclean
apt-get clean
#32 bit architecture support:: not required for 32 bit template
if [ "${arch}" != "i386" ]; then
dpkg --add-architecture i386
apt-get update
${apt_get} install links:i386 libuuid1:i386 libc6:i386
fi
# Install xenserver guest utilities as debian repos don't have it
wget https://mirrors.kernel.org/ubuntu/pool/universe/x/xe-guest-utilities/xe-guest-utilities_7.4.0-0ubuntu1_amd64.deb
dpkg -i xe-guest-utilities_7.4.0-0ubuntu1_amd64.deb

View File

@ -0,0 +1,82 @@
{
"_license": "Apache License 2.0",
"description": "CloudStack SystemVM template",
"provisioners": [
{
"type": "shell",
"execute_command": "echo 'cloud'|sudo -S bash '{{.Path}}'",
"scripts": [
"scripts/apt_upgrade.sh",
"scripts/configure_grub.sh",
"scripts/configure_locale.sh",
"scripts/configure_networking.sh",
"scripts/configure_acpid.sh",
"scripts/install_systemvm_packages.sh",
"scripts/configure_conntrack.sh",
"../cloud_scripts_shar_archive.sh",
"scripts/configure_systemvm_services.sh",
"scripts/authorized_keys.sh",
"scripts/configure_persistent_config.sh",
"scripts/configure_login.sh",
"scripts/cleanup.sh",
"scripts/finalize.sh"
]
}
],
"builders": [
{
"type": "qemu",
"accelerator": "kvm",
"headless": true,
"qemuargs": [
[ "-m", "512M" ],
[ "-smp", "cpus=1,maxcpus=1,cores=1" ]
],
"disk_size": 2100,
"format": "qcow2",
"disk_interface": "virtio",
"net_device": "virtio-net",
"iso_url": "https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.3.0-amd64-netinst.iso",
"iso_checksum": "8775231d6f56a3d8f116eb64fe048f5cbd2ea0f8c092a1cb7608bcb4106f9c85cb69ce68f53bd381019ab40f1c0316843036daf3fd9107c81c58a240334cc747",
"iso_checksum_type": "sha512",
"vm_name": "systemvmtemplate",
"output_directory": "../dist",
"http_directory": "http",
"http_port_min": 10082,
"http_port_max": 10089,
"ssh_host_port_min": 2222,
"ssh_host_port_max": 2229,
"ssh_username": "cloud",
"ssh_password": "cloud",
"ssh_port": 22,
"ssh_wait_timeout": "30m",
"boot_wait": "10s",
"boot_command": [
"<esc><wait>",
"install <wait>",
" preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg <wait>",
" debian-installer=en_US <wait>",
" auto <wait>",
" locale=en_US <wait>",
" kbd-chooser/method=us <wait>",
" netcfg/get_hostname=systemvm <wait>",
" netcfg/get_domain=apache.org <wait>",
" fb=false <wait>",
" debconf/frontend=noninteractive <wait>",
" console-setup/ask_detect=false <wait>",
" console-keymaps-at/keymap=us <wait>",
" keyboard-configuration/xkb-keymap=us <wait>",
"<enter><wait>"
],
"shutdown_command": "sudo halt -p"
}
]
}

View File

@ -1,180 +0,0 @@
#!/bin/bash -xl
# note: the -l is needed here for bash to always make a login shell and load rvm if it hasn't been loaded
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# test script for build.sh which tries a variety of options/configs to make different vms
set -e
DEBUG="${DEBUG:-}"
TRACE="${TRACE:-0}"
###
### Configuration
###
if [[ "${DEBUG}" == "1" ]]; then
set -x
fi
# which test to run
test_to_run=${1:-}
# build.sh settings for running the tests
appliance=debianbase
version=`date "+%Y%m%d%H%M%S"`
branch=`git status | grep '# On branch' | awk '{print $4}'`
BUILD_NUMBER="${BUILD_NUMBER:-}"
ssh_key=
# where we are running the tests from
CURR_DIR=${PWD}
# where this script is
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
# ensure we are running in isolation
if [ ${CURR_DIR} == ${SCRIPT_DIR} ]; then
mkdir -p ../appliance-work
cd ../appliance-work
CURR_DIR=${PWD}
fi
###
### testing 'framework'
###
function test_result() {
log INFO "$@"
add_on_exit log INFO "$@"
}
function run_test() {
set +e
cleanup
fixture
log INFO running test: "$@"
eval $@
result=$?
if ${result}; then
test_result "$@" FAIL
else
test_result "$@" OK
fi
cleanup
set -e
}
function cleanup() {
(
cd ${CURR_DIR};
rm -rf iso definitions Gemfile shar_cloud_scripts convert_ovf_vbox_to_esx.xslt .rvmrc;
)
}
function fixture() {
(
cd ${CURR_DIR};
mkdir -p ${SCRIPT_DIR}/iso;
ln -s ${SCRIPT_DIR}/iso;
mkdir definitions;
ln -s ${SCRIPT_DIR}/definitions/${appliance} definitions/${appliance};
ln -s ${SCRIPT_DIR}/Gemfile;
ln -s ${SCRIPT_DIR}/shar_cloud_scripts.sh;
ln -s ${SCRIPT_DIR}/convert_ovf_vbox_to_esx.xslt;
ln -s ${SCRIPT_DIR}/.rvmrc;
)
}
###
### Test definitions
###
function do_test_vm() {
prepare
create_definition
veewee_build
retry 10 check_appliance_shutdown
retry 10 remove_shares
veewee_destroy
}
function do_test_export() {
prepare
create_definition
veewee_build
retry 10 check_appliance_shutdown
retry 10 remove_shares
# Get appliance uuids
local vm_info=`vboxmanage showvminfo "${appliance_build_name}"`
local machine_uuid=`echo "${vm_info}" | grep UUID | head -1 | awk '{print $2}'`
local hdd_uuid=`echo "${vm_info}" | grep vdi | head -1 | awk '{print $8}' | cut -d ')' -f 1`
local hdd_path=`vboxmanage list hdds | grep "${appliance_build_name}\/" | grep vdi | \
cut -c 14- | sed ${sed_regex_option} 's/^ *//'`
compact_hdd "${hdd_uuid}"
xen_server_export "${hdd_path}"
kvm_export "${hdd_path}"
vmware_export "${machine_uuid}" "${hdd_uuid}"
hyperv_export "${hdd_uuid}"
veewee_destroy
}
function test_basic_veewee_invocation() {
appliance=debianbase
appliance_build_name=${appliance}${branch_tag}${version_tag}
do_test_vm
}
function test_export() {
appliance=debianbase
appliance_build_name=${appliance}${branch_tag}${version_tag}
do_test_export
}
function test_systemvm() {
appliance=systemvmtemplate
appliance_build_name=${appliance}${branch_tag}${version_tag}
do_test_vm
}
function test_systemvm64() {
appliance=systemvm64template
appliance_build_name=${appliance}${branch_tag}${version_tag}
do_test_vm
}
function test_suite() {
if [ "${test_to_run}" == "" ]; then
# list of all tests goes here
run_test test_basic_veewee_invocation
run_test test_systemvm
run_test test_systemvm64
run_test test_export
else
run_test "${test_to_run}"
fi
}
###
### Main invocation
###
source ${SCRIPT_DIR}/build.sh
return 2>/dev/null || test_suite

View File

@ -1,49 +0,0 @@
#!/usr/bin/env ruby
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
lines = `VBoxManage list hdds`
disks = lines.split(/\n\s*\n/)
disks.each do |disk|
disk_lines = disk.split(/\n/)
disk_config = {}
disk_lines.each do |line|
pair = line.split(/:\s*/)
disk_config[pair[0]] = pair[1]
# if pair[0] == 'Location'
# location = pair[1]
# if location.include? '/Snapshots/'
# disk_config['is_snapshot'] = true
# end
# if location.include? '/VirtualBox VMs/'
# disk_config['vm_name'] = location.split('/VirtualBox VMs/')[1].split('/')[0]
# disk_config['disk_name'] = location.split('/')[-1]
# disk_config['is_virtualbox_vm'] = true
# else
# disk_config['is_virtualbox_vm'] = false
# disk_config['disk_name'] = location.split('/')[-1]
# end
# end
end
if disk_config.include? 'Location'
cmd="VBoxManage closemedium disk '#{disk_config['Location']}' --delete"
puts cmd
`#{cmd}`
end
end

View File

@ -1,77 +0,0 @@
#!/usr/bin/env ruby
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# script that tries hard to forcibly shut down all vms
# gem install sys-proctable
require 'sys/proctable'
include Sys
do_delete = (ARGV.include? 'delete' or ARGV.include? '--delete' or ARGV.include? '-d')
do_kill = (ARGV.include? 'kill' or ARGV.include? '--kill' or ARGV.include? '-k')
lines = `VBoxManage list vms`
vms = lines.split(/\n/)
if vms.nil?
vms = []
end
vms.each do |vmline|
vm_info = /\"(.*)\"[^{]*\{(.*)\}/.match(vmline)
next if vm_info.nil?
vm_name = vm_info[1]
vm_uuid = vm_info[2]
cmd="VBoxManage controlvm #{vm_name} poweroff"
puts cmd
`#{cmd}`
if do_delete
sleep(1)
cmd="VBoxManage unregistervm #{vm_name} --delete"
puts cmd
`#{cmd}`
end
if do_kill
sleep(1)
# ps x | grep VBoxHeadless | grep systemvm64template-4.4.0 | egrep -o '^\s*[0-9]+' | xargs kill
ProcTable.ps do |p|
next unless p.cmdline.include? "VBoxHeadless"
next unless p.cmdline.include? vm_name
# not all rubies / proctables expose ruid
if defined? p.ruid
# VBoxManage should only list _our_ vms, but just to be safe...
next unless p.ruid == Process.uid
end
puts "kill -SIGKILL #{p.pid}"
begin
Process.kill("KILL", p.pid)
rescue => exception
puts exception.backtrace
end
sleep(5)
puts "kill -SIGTERM #{p.pid}"
begin
Process.kill("TERM", p.pid)
rescue => exception
puts exception.backtrace
end
end
end
end

View File

@ -1,206 +0,0 @@
# -*- mode: ruby -*-
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#
# vi: set ft=ruby :
# Vagrantfile API/syntax version. Don't touch unless you know what you're doing!
VAGRANTFILE_API_VERSION = "2"
$mysql_script = <<SCRIPT
hostname db
echo nameserver 8.8.8.8 > /etc/resolv.conf
yum install mysql-server -y
service mysqld start
chkconfig mysqld on
/usr/bin/mysqladmin -u root password 'cloud'
mysql -uroot -pcloud -e "use mysql;GRANT ALL ON *.* to root@'%' IDENTIFIED BY 'cloud' WITH GRANT OPTION; FLUSH PRIVILEGES;"
mkdir -p /opt/storage/secondary
echo "/opt/storage/secondary *(rw,no_root_squash)" > /etc/exports
service nfs start
mkdir -p /opt/storage/secondary/template/tmpl/1/1
cd /opt/storage/secondary/template/tmpl/1/1
cp /vagrant/templates/tmpl/1/1/systemvmtemplate.vhd.bz2 .
bzip2 -d systemvmtemplate.vhd.bz2
mv systemvmtemplate.vhd edd5d5e5-b363-4926-a85b-d742ddd4a801.vhd
checksum=$(md5sum edd5d5e5-b363-4926-a85b-d742ddd4a801.vhd | cut -f 1 -d " ")
cat > template.properties <<TEMPLATE
filename=edd5d5e5-b363-4926-a85b-d742ddd4a801.vhd
vhd=true
id=1209
vhd.filename=edd5d5e5-b363-4926-a85b-d742ddd4a801.vhd
public=true
uniquename=routing-1
vhd.virtualsize=2097152000
virtualsize=2097152000
checksum=${checksum}
hvm=false
description=SystemVM Template (XenServer)
vhd.size=2101252608
size=2101252608
TEMPLATE
mkdir -p /opt/storage/secondary/template/tmpl/1/5
cd /opt/storage/secondary/template/tmpl/1/5
cp /vagrant/templates/tmpl/1/5/ce5b212e-215a-3461-94fb-814a635b2215.vhd .
cp /vagrant/templates/tmpl/1/5/template.properties .
SCRIPT
$xen_script = <<SCRIPT
hostname xen
xe pif-scan host-uuid=$(xe host-list --minimal)
xe pif-plug uuid=$(xe pif-list device=eth1 --minimal)
xe network-param-set name-label=xenbr0 uuid=$(xe network-list bridge=xenbr0 --minimal)
xe network-param-set name-label=xenbr1 uuid=$(xe network-list bridge=xenbr1 --minimal)
xe pif-reconfigure-ip mode=static uuid=$(xe pif-list device=eth1 --minimal) IP=192.168.56.234 netmask=255.255.255.0
xe host-management-reconfigure pif-uuid=$(xe pif-list device=eth1 --minimal)
SCRIPT
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
# All Vagrant configuration is done here. The most common configuration
# options are documented and commented below. For a complete reference,
# please see the online documentation at vagrantup.com.
# Every Vagrant virtual environment requires a box to build off of.
config.vm.define "xen" do |xen|
xen.vm.box = "duffy/xenserver"
xen.vm.provision "shell", inline: $xen_script
end
config.vm.define "db" do |db|
db.vm.box = "chef/centos-6.5"
db.vm.network "private_network", ip: "192.168.56.94"
db.vm.network "forwarded_port", guest: 3306, host: 3306
db.vm.provision "shell", inline: $mysql_script
end
# Disable automatic box update checking. If you disable this, then
# boxes will only be checked for updates when the user runs
# `vagrant box outdated`. This is not recommended.
# config.vm.box_check_update = false
# Create a forwarded port mapping which allows access to a specific port
# within the machine from a port on the host machine. In the example below,
# accessing "localhost:8080" will access port 80 on the guest machine.
# config.vm.network "forwarded_port", guest: 80, host: 8080
# Create a private network, which allows host-only access to the machine
# using a specific IP.
# config.vm.network "private_network", ip: "192.168.33.10"
# Create a public network, which generally matched to bridged network.
# Bridged networks make the machine appear as another physical device on
# your network.
# config.vm.network "public_network"
# If true, then any SSH connections made will enable agent forwarding.
# Default value: false
# config.ssh.forward_agent = true
# Share an additional folder to the guest VM. The first argument is
# the path on the host to the actual folder. The second argument is
# the path on the guest to mount the folder. And the optional third
# argument is a set of non-required options.
# config.vm.synced_folder "../data", "/vagrant_data"
# Provider-specific configuration so you can fine-tune various
# backing providers for Vagrant. These expose provider-specific options.
# Example for VirtualBox:
#
# config.vm.provider "virtualbox" do |vb|
# # Don't boot with headless mode
# vb.gui = true
#
# # Use VBoxManage to customize the VM. For example to change memory:
# vb.customize ["modifyvm", :id, "--memory", "1024"]
# end
#
# View the documentation for the provider you're using for more
# information on available options.
# Enable provisioning with CFEngine. CFEngine Community packages are
# automatically installed. For example, configure the host as a
# policy server and optionally a policy file to run:
#
# config.vm.provision "cfengine" do |cf|
# cf.am_policy_hub = true
# # cf.run_file = "motd.cf"
# end
#
# You can also configure and bootstrap a client to an existing
# policy server:
#
# config.vm.provision "cfengine" do |cf|
# cf.policy_server_address = "10.0.2.15"
# end
# Enable provisioning with Puppet stand alone. Puppet manifests
# are contained in a directory path relative to this Vagrantfile.
# You will need to create the manifests directory and a manifest in
# the file default.pp in the manifests_path directory.
#
# config.vm.provision "puppet" do |puppet|
# puppet.manifests_path = "manifests"
# puppet.manifest_file = "site.pp"
# end
# Enable provisioning with chef solo, specifying a cookbooks path, roles
# path, and data_bags path (all relative to this Vagrantfile), and adding
# some recipes and/or roles.
#
# config.vm.provision "chef_solo" do |chef|
# chef.cookbooks_path = "../my-recipes/cookbooks"
# chef.roles_path = "../my-recipes/roles"
# chef.data_bags_path = "../my-recipes/data_bags"
# chef.add_recipe "mysql"
# chef.add_role "web"
#
# # You may also specify custom JSON attributes:
# chef.json = { mysql_password: "foo" }
# end
# Enable provisioning with chef server, specifying the chef server URL,
# and the path to the validation key (relative to this Vagrantfile).
#
# The Opscode Platform uses HTTPS. Substitute your organization for
# ORGNAME in the URL and validation key.
#
# If you have your own Chef Server, use the appropriate URL, which may be
# HTTP instead of HTTPS depending on your configuration. Also change the
# validation key to validation.pem.
#
# config.vm.provision "chef_client" do |chef|
# chef.chef_server_url = "https://api.opscode.com/organizations/ORGNAME"
# chef.validation_key_path = "ORGNAME-validator.pem"
# end
#
# If you're using the Opscode platform, your validator client is
# ORGNAME-validator, replacing ORGNAME with your organization name.
#
# If you have your own Chef Server, the default validation client name is
# chef-validator, unless you changed the configuration.
#
# chef.validation_client_name = "ORGNAME-validator"
end

View File

@ -1 +0,0 @@
Please put your systemVM into this location with the name: systemvmtemplate.vhd.bz2

View File

@ -1,2 +0,0 @@
Please put your tinyVHD image into this location with the name: ce5b212e-215a-3461-94fb-814a635b2215.vhd
Also put the template.properties file for the tinyVHD image into this location.

View File

@ -1,52 +0,0 @@
*.gem
*.rbc
.bundle
.config
coverage
InstalledFiles
lib/bundler/man
pkg
rdoc
spec/reports
test/tmp
test/version_tmp
tmp
# YARD artifacts
.yardoc
_yardoc
doc/
# OSX files
.DS_Store
.AppleDouble
.LSOverride
# Icon must end with two \r
Icon
# Thumbnails
._*
# Files that might appear on external disk
.Spotlight-V100
.Trashes
# Directories potentially created on remote AFP share
.AppleDB
.AppleDesktop
Network Trash Folder
Temporary Items
.apdisk
# Vagrant stuff
boxes/*
.vagrant
# Systemvm ISO
systemvm.iso
iso/*
rspec.xml
vendor/

View File

@ -1 +0,0 @@
1.9.3-p484

View File

@ -1,24 +0,0 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
rvm use ruby-1.9.3@vagrant-release-cloudstack --create
export VAGRANT_HOME=$HOME/.vagrant.d-release-cloudstack
bundle check > /dev/null 2>&1
RETVAL=$?
if [ $RETVAL -ne 0 ]; then
bundle install
fi

View File

@ -1,30 +0,0 @@
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
===========================================================
Allows spinning up the systemvm appliance from ../../appliance inside
vagrant, and then running tests against it with nose.
To use, install vagrant, rvm, ruby, bundler, python and pip.
Then run ./test.sh.
To write tests, create files underneath ../../../test/systemvm
named test_xxx.py. These tests are standard python unit tests with
some logic to SSH into the SystemVM. See
../../../test/systemvm/README.md for more info.

View File

@ -1,41 +0,0 @@
#!/bin/bash
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# In some cases, while booting a virtual machine, an IDE controller
# will be created for it. It seems that the VirtualBox GUI likes doing
# this: when a particular machine has booted at least once with its
# GUI turned on, this will happen pretty consistently.
#
# Having an IDE controller and a SATA controller breaks the assumptions
# in the systemvm scripts about what disks are attached, causing it to
# not find the systemvm.iso.
#
# So, we delete the IDE controller using Vagrant.
#
# Unfortunately, when the IDE controller does not exist, that deletion
# fails, causing vagrant to fail. To work around this, we inject this
# script into the path, causing vagrant to try to continue booting.
/usr/bin/VBoxManage "$@"
exitcode=$?
if [[ "$1" == "storagectl" ]]; then
exit 0
else
exit ${exitcode}
fi

View File

@ -1,123 +0,0 @@
#-*- mode: ruby -*-
# vi: set ft=ruby :
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
include RbConfig
basedir = File.dirname(__FILE__)
VAGRANTFILE_API_VERSION = '2'
unless ENV['VPC_IP']
STDERR.puts 'You did not specify the VPC IP by settings the VPC_IP environment variable'
STDERR.puts 'Using the default VPC_IP=192.168.56.30'
end
VPC_IP = ENV['VPC_IP'] || '192.168.56.30'
VPC_NAME='r-' + VPC_IP.split('.').last + '-VM'
if ARGV[0] == 'up'
iso_util=''
case CONFIG['host_os']
when /mswin|windows/i
STDERR.puts 'Windows is not supported'
exit 1
when /linux|arch/i
iso_util = "mkisofs -J -o #{basedir}/systemvm.iso #{basedir}/iso"
when /sunos|solaris/i
STDERR.puts 'Solaris is not supported'
exit 1
when /darwin/i
iso_util = "hdiutil makehybrid -iso -joliet -o #{basedir}/systemvm.iso #{basedir}/iso/"
else
STDERR.puts 'This OS is not supported'
exit 1
end
system "rm -rf #{basedir}/systemvm.iso"
system "mkdir -p #{basedir}/iso/"
unless File.exist? "#{basedir}/../../../systemvm/dist/cloud-scripts.tgz"
STDERR.puts 'No cloud-scripts.tgz found. Did you run the maven build?'
exit 1
end
system "cp #{basedir}/../../../systemvm/dist/cloud-scripts.tgz #{basedir}/iso/"
unless File.exist? "#{basedir}/../../../systemvm/dist/systemvm.zip"
STDERR.puts 'No systemvm.zip found. Did you run the maven build?'
exit 1
end
system "cp #{basedir}/../../../systemvm/dist/systemvm.zip #{basedir}/iso/"
system "cp #{basedir}/vagrant.pub #{basedir}/iso/authorized_keys"
system 'chmod 600 iso/authorized_keys'
system iso_util
end
Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|
config.vm.box = 'cloudstack/systemvm'
config.vm.network 'private_network', ip: VPC_IP, auto_config: false
config.vm.network 'private_network', ip: '192.168.56.50' #, auto_config: false
config.vm.network 'private_network', ip: '192.168.56.51' #, auto_config: false
config.vm.network 'private_network', ip: '192.168.56.52' #, auto_config: false
config.vm.synced_folder 'vagrant', '/vagrant', disabled: true
#noinspection RubyStringKeysInHashInspection
patches = {
'config/opt' => '/opt',
'config/root' => '/root',
'config/var' => '/var',
'config/etc/iptables' => '/etc/iptables',
# cannot have two rsyncs pointing to the same dir
# 'vpn/etc' => '/etc',
# 'vpn/opt' => '/opt',
'xe' => '/usr/sbin'
}
patches.each_pair do |patch, dest|
config.vm.synced_folder(
"#{basedir}/../../../systemvm/patches/debian/#{patch}",
dest,
type: 'rsync',
rsync__chown: false,
rsync__args: %w(--verbose --archive --exclude=authorized_keys) # no --delete!
)
end
config.ssh.forward_agent = true
config.ssh.username = 'root'
config.ssh.host = VPC_IP
config.ssh.port = 3922
config.ssh.guest_port = 3922
config.vm.provider 'virtualbox' do |vb|
# enable or disable headless mode
vb.gui = false
vb.customize ['modifyvm', :id, '--memory', '256']
vb.customize ['storagectl', :id, '--name', 'IDE Controller', '--remove']
vb.customize ['storageattach', :id, '--storagectl', 'SATA Controller', '--port', '1', '--type', 'dvddrive',
'--medium', './systemvm.iso']
vb.customize('pre-boot', ['modifyvm', :id, '--nic1', 'none'])
extra_data='cmdline:console=hvc0 vpccidr=172.16.0.0/16 domain=devcloud.local dns1=8.8.8.8 dns2=8.8.8.4' +
" template=domP name=#{VPC_NAME} eth0ip=#{VPC_IP}" +
' eth0mask=255.255.255.0 type=vpcrouter disable_rp_filter=true'
vb.customize('pre-boot', ['setextradata', :id, 'VBoxInternal/Devices/pcbios/0/Config/DmiOEMVBoxRev', extra_data])
vb.customize ['modifyvm', :id, '--nic1', 'hostonly', '--hostonlyadapter1', 'vboxnet0']
vb.customize ['modifyvm', :id, '--nic2', 'hostonly', '--hostonlyadapter2', 'vboxnet0']
vb.customize ['modifyvm', :id, '--nic3', 'hostonly', '--hostonlyadapter3', 'vboxnet0']
end
end

View File

@ -1,213 +0,0 @@
#!/bin/bash -l
# note: the -l is needed here for bash to always make a login shell and load rvm if it hasn't been loaded
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# build script which wraps around nose to test the systemvm
function usage() {
cat <<END
Usage:
./build.sh
END
exit 0
}
echo $@ | grep help >/dev/null && usage
echo $@ | grep '\-h' >/dev/null && usage
set -e
###
### Configuration
###
# whether to show DEBUG logs
DEBUG="${DEBUG:-}"
# whether to have other commands trace their actions
TRACE="${TRACE:-0}"
JENKINS_HOME=${JENKINS_HOME:-}
if [[ ! -z "${JENKINS_HOME}" ]]; then
DEBUG=1
fi
VPC_IP="${VPC_IP:-192.168.56.254}"
export VPC_IP
# inject our custom VBoxManage wrapper script
export PATH=$PWD:$PATH
###
### Generic helper functions
###
# how to tell sed to use extended regular expressions
os=`uname`
sed_regex_option="-E"
if [ "${os}" == "Linux" ]; then
sed_regex_option="-r"
fi
# logging support
if [[ "${DEBUG}" == "1" ]]; then
set -x
fi
function log() {
local level=${1?}
shift
if [[ "${DEBUG}" != "1" && "${level}" == "DEBUG" ]]; then
return
fi
local code=
local line="[$(date '+%F %T')] $level: $*"
if [ -t 2 ]
then
case "$level" in
INFO) code=36 ;;
DEBUG) code=30 ;;
WARN) code=33 ;;
ERROR) code=31 ;;
*) code=37 ;;
esac
echo -e "\033[${code}m${line}\033[0m"
else
echo "$line"
fi >&2
}
function error() {
log ERROR $@
exit 1
}
# cleanup code support
declare -a on_exit_items
function on_exit() {
for (( i=${#on_exit_items[@]}-1 ; i>=0 ; i-- )) ; do
log DEBUG "on_exit: ${on_exit_items[${i}]}"
eval ${on_exit_items[${i}]}
done
}
function add_on_exit() {
local n=${#on_exit_items[*]}
on_exit_items[${n}]="$*"
if [ ${n} -eq 0 ]; then
log DEBUG "Setting trap"
trap on_exit EXIT
fi
}
# retry code support
function retry() {
local times=$1
shift
local count=0
while [ ${count} -lt ${times} ]; do
"$@" && break
count=$(( $count + 1 ))
sleep ${count}
done
if [ ${count} -eq ${times} ]; then
error "Failed ${times} times: $@"
fi
}
###
### Script logic
###
function setup_ruby() {
local bundle_args=
if [[ ! -z "${JENKINS_HOME}" ]]; then
# inspired by https://github.com/CloudBees-community/rubyci-clickstart/blob/master/bin/run-ci
# also see https://rvm.io/integration/jenkins
# .rvmrc won't get trusted/auto-loaded by jenkins by default
export VAGRANT_HOME=$HOME/.vagrant.d-release-cloudstack
# rvm use ruby-1.9.3@vagrant-release-cloudstack --create
# do not use --deployment since that requires Gemfile.lock...and we prefer an up-to-date veewee
bundle_args="--path vendor/bundle"
fi
bundle check || bundle install ${bundle_args}
}
function setup_python() {
which pip || sudo easy_install pip
pip install nose paramiko python-vagrant envassert cuisine fabric
}
function prepare() {
log INFO "preparing for build"
setup_ruby
setup_python
rm -f systemvm.iso
}
function box_update() {
log INFO "invoking vagrant box update"
vagrant box update
log INFO "vagrant box update complete"
}
function vagrant_up() {
log INFO "invoking vagrant up"
vagrant up --no-provision
log INFO "vagrant up complete"
}
function vagrant_provision() {
log INFO "invoking vagrant provision"
vagrant provision
log INFO "vagrant up complete"
}
function nose() {
log INFO "invoking nose"
PWD=`pwd`
(cd ../../../test/systemvm;
mkdir -p target/test-reports;
nosetests --with-xunit --xunit-file=target/test-reports/xunit.xml;)
}
function vagrant_destroy() {
log INFO "invoking vagrant destroy"
vagrant destroy -f
log INFO "vagrant destroy complete"
}
###
### Main invocation
###
function main() {
prepare
box_update
vagrant_destroy
add_on_exit vagrant_destroy
vagrant_up
vagrant_provision
nose
add_on_exit log INFO "BUILD SUCCESSFUL"
exit
}
# we only run main() if not source-d
return 2>/dev/null || main

View File

@ -1 +0,0 @@
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key