86 Commits

Author SHA1 Message Date
Rohit Yadav
f8385a48da
Add systemvmtemplate arm64 build support (#9437)
* Move and copy files to new filenames

* Adjust bash scripts to support arm64

* change preseed for arm64 to create EFI partition

* change packer template files to support arm64

* Correct preseed file name in x86_64 packer template

* fixes

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* try different machine cpu

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* don't accelerate KVM for different arch (arm64 vm on amd64 host)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* fix building arm64 on x86

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* Update README.md

---------

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Co-authored-by: StepBee <stephan@bienek.org>
2024-07-26 13:38:03 +05:30
Spaceman1984
27e7637b02
Updated vmware virtual hardware version in systemvmtemplate build script (#3921)
Updated vmware virtual hardware version in systemvmtemplate build script
2020-05-30 17:12:34 +05:30
Rohit Yadav
d90341ebf1
cloudstack: add JDK11 support (#3601)
This adds support for JDK11 in CloudStack 4.14+:

- Fixes code to build against JDK11
- Bump to Debian 9 systemvmtemplate with openjdk-11
- Fix Travis to run smoketests against openjdk-11
- Use maven provided jdk11 compatible mysql-connector-java
- Remove old agent init.d scripts

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2020-02-12 12:58:25 +05:30
Rohit Yadav
ba61b512b2 [Merge 4.11] CLOUDSTACK-10341: Reduce systemvmtemplate size (#2506)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-28 13:21:59 +05:30
Rohit Yadav
986ecfa7e9
CLOUDSTACK-10341: Reduce systemvmtemplate size, install nftables (#2506)
This reduces systemvmtemplate size by 600MB and installs nftables,
updates iptables. This also fixes a failing smoke test.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-28 13:11:51 +05:30
Khosrow Moossavi
1708838518 CLOUDSTACK-10268: Fix and enhance package script (#2433)
- new flag `-T, --use-timestamp` to use `timestamp` when POM version contains SNAPSHOT
  - in the final artifacts (jar) name
  - in the final package (rpm, deb) name
  - in `/etc/cloudstack-release` file of SystemVMs
  - in the Management Server > About dialog
- if there's a "branding" string in the POM version (e.g. `x.y.z.a-NAME[-SNAPSHOT]`),
the branding name will be used in the final generated pacakge name such as following:
  - `cloudstack-management-x.y.z.a-NAME.NUMBER.el7.centos.x86_64`
  - `cloudstack-management_x.y.z.a-NAME-NUMBER~xenial_all.deb`
- branding string can be overriden with newly added `-b, --brand` flag
- handle the new format version for VR version
- fix long opts (they were broken)
- tolerate and show a warning message for unrecognized flags
- usage help reformat

* Deprecate Version class in favor or CloudStackVersion
2018-03-15 10:55:29 +00:00
Rohit Yadav
3839239a21 CLOUDSTACK-10194: Use packer for building systemvmtemplates
- This migrates the current systemvmtemplate build system from
  veewee/virtualbox to packer and qemu based.
- This also introduces and updates a CentOS7 built-in template.
- Remove old appliance build scripts and files.
- Adds iftop package (CLOUDSTACK-9785)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 09:22:44 +05:30
Rohit Yadav
facc5945f0 CLOUDSTACK-10193: Fix smoke tests failures with new systemvmtemplate
- Several systemvmtemplate optimizations
- Uses new macchinina template for running smoke tests
- Switch to latest Debian 9.3.0 release for systemvmtemplate
- Introduce a new `get_test_template` that uses tiny test template
  such as macchinina as defined test_data.py
- rVR related fixes and improvements

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 09:22:44 +05:30
Rohit Yadav
78bb74188d CLOUDSTACK-10013: Migrate systemvmtemplate to Debian9
SystemVM changes to work on Debian 9
- Migrate away from chkconfig to systemctl
- Remove xenstore-utils override deb pkg
- Fix runlevel in sysv scripts for systemd

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 09:22:44 +05:30
Rohit Yadav
92fd5bee3d CLOUDSTACK-9591: Fix systemvmtemplate to not include network details
This removes nic/network specific details while exporting the systemvmtemplate
for vmware (ova file). Having this causes the ssvms to not deploy in
dvswitch-based vmware environments that have no vswitch portgroups (dummy etc).
Tested this on a local Trillian env.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-03-30 13:47:53 +05:30
Rohit Yadav
2cddaf3d36 CLOUDSTACK-9447: fix build and upgrade to debian 7.11 iso
- Update base debian iso to version 7.11
- Upgrade ruby version to 2.3.0 (latest/stable)
- Fix Gemfile
- Update README
- Fix openswan pkg name with the same version
- Remove cloud-cleanup it's not available

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-08-05 12:40:07 +05:30
Rohit Yadav
0f82650dcf systemvmtemplate: use ovftool for building vmware template if available
This commit fixes a chmod issue where extracted vmdk file is not readable by all
users/groups. The other improvement is to use ovftool to build systemvm template
for vmware if it's available. This is based on a dev ML discussion and a suggested
approach by Ilya: http://markmail.org/message/kntsetgxdbppfh22

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-05-12 16:46:23 +02:00
Rohit Yadav
8fe3c0b8ba appliance: remove raw.img before exporting and building template for KVM
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-25 17:27:03 +05:30
Rohit Yadav
7a2e27c3b3 appliance: save md5 checksums after building systemvm templates
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-20 15:16:57 +05:30
Funs
c27c69438b hypervisors: add OVM3 plugin that supports OVM 3.2.1/3.3.x
This is a plugin that puts in ovm3 support ranging from 3.3.1 to 3.3.2. Basic
functionality is in here, advanced networking etc..

Snapshots only work when a VM is stopped now due to the semantics of OVM's raw
image implementation (so snapshots should work on a storage level underneath the
hypervisor shrug)

This closes #113

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-03-12 11:33:42 +05:30
Rohit Yadav
6b6ce5d074 appliance: don't fail is bak is not found
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-01-28 18:27:19 +05:30
Will Stevens
03daae571b CLOUDSTACK-7959: fixed 'qemu-img convert' compat issue 2014-12-02 12:15:16 -05:00
Will Stevens
8e689b1148 Updated the system vm build to remove incompatible qemu-img 'compat' option 2014-11-21 05:59:06 -05:00
Edison Su
05bec59c14 CS-27148 system vm image build process, needs to build an old version of qemu image, otherwise, it won't work on RHEL 6 Reviewed-by:Frank 2014-11-06 15:25:57 -08:00
Harikrishna Patnala
34b7288ca4 CLOUDSTACK-7748: Cloudstack version number is missing in system vm template. /etc/cloudstack-release in system vm does not have version number which leads in vm deployment failure while checking router version. 2014-11-03 12:13:04 +05:30
Rohit Yadav
cb5597167d appliance: build and export raw systemvm image for OVM
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2014-09-22 21:58:25 +02:00
Leo Simons
ba009ed51a CLOUDSTACK-7143: attempt to be safer when cleaning up 2014-09-22 21:38:17 +02:00
Leo Simons
362946c52e CLOUDSTACK-7143: don't call rvm from build
It is simpler to expect that rvm setup is done outside of this build.
The buildacloud.org jenkins has rvm installed/enabled by default so
does not invoke rvm.
2014-09-22 21:38:17 +02:00
Leo Simons
a8500ed73e CLOUDSTACK-7143: survive / in branch names 2014-09-22 21:38:17 +02:00
Leo Simons
115b2cf8e7 CLOUDSTACK-7143: only archive the zipped hyperv image
Cherry-pick of d09acc44f2b36cd593ebe97ab1673b7757ea2c25.
2014-09-22 21:38:17 +02:00
Leo Simons
d56e950e85 CLOUDSTACK-7143: work on vagrant box export
Running --export creates the .ovf and the .vmdk files referenced
from that .ovf in one go. Guessing/predicting the names of the .vmdk
files is not fool-proof.
2014-09-22 21:38:17 +02:00
Ian Southam
cab4dd8258 CLOUDSTACK-7143: Fixed box file for virtualbox 2014-09-22 21:38:16 +02:00
Leo Simons
598c472b73 CLOUDSTACK-7143: remove erroneous backticks
The backticks in the Vagrantfile template were getting evaluated by bash.
This caused some harmless but confusing error messages to appear on running
the build. Easy fix is to remove them.
2014-09-22 21:38:16 +02:00
Leo Simons
35347f8350 CLOUDSTACK-7143: try to fix jenkins problems with last 2 commits
* bundle install needs to run before running the vbox cleaning scripts,
  so move prepare step before clean step
* feature branches have / in their name which is a bad character to
  put into filenames
2014-09-22 21:38:16 +02:00
Leo Simons
00b39de0f8 CLOUDSTACK-7143: more reliable support for vagrant box export
Veewee supports exporting vagrant boxes out of virtualbox, out of the box.
However, it assumes that it can export a disk if the shutdown of the vm that
is using that disk has succeeded. This assumption is not strictly always true
(see previous commit). So, we replicate the bit of logic in veewee for making
vagrant boxes.

This has the added side benefit of creating an .ovf export only once, rather
than once for vmware and then again for vagrant.
2014-09-22 21:38:16 +02:00
Leo Simons
be8b2d7c21 CLOUDSTACK-7143: more robust box cleanup
Having experimented with many edge cases of running multiple build.sh
commands in parallel / against busy virtualbox setups, the only really
reliable way to produce consistent images is to not do these commands
in parallel and to not do them while the machine is doing many other
things.

If virtualbox or the machine that hosts it is very busy, and/or it has
a lot of disks it knows/knew about, and/or its tuesday, behavior may
be a bit different.

Realizing this reality, this commit adds some scripts that try really
hard to set virtualbox back to known/healthy state before building.
2014-09-22 21:38:16 +02:00
Leo Simons
d658fc4637 CLOUDSTACK-7143: attempt to export a vagrant box 2014-09-22 21:38:16 +02:00
Leo Simons
25bb682cd1 CLOUDSTACK-7143: fix the fix for rvm/ruby/bundler setup on jenkins
The recommended use of --deployment goes along with the requirement to
check Gemfile.lock into source control, which I don't want to do.
2014-09-22 21:38:15 +02:00
Leo Simons
a56392b5e4 CLOUDSTACK-7143: attempt at a reliable way to set up rvm/bundler on jenkins 2014-09-22 21:38:14 +02:00
Leo Simons
a7d07bf3df CLOUDSTACK-7143: fix a bug where a script file rename wasn't propagated to build.sh 2014-09-22 21:38:14 +02:00
Leo Simons
dd34d10159 CLOUDSTACK-7143: add tests for the appliance build
Create a new minimal 'debianbase' definition which is a veewee template
that's a lot like the systemvmtemplate, but does not have any
systemvm-ness in it. Use it to create a new test.sh which tests a few
common invocations of build.sh work as desired.

This is mainly useful for debugging whether the appliance build process
is working / consistent; in order to test a systemvm itself it should
really first be merged with systemvm.iso.
2014-09-22 21:38:14 +02:00
Leo Simons
33fd6894a0 CLOUDSTACK-7143: use shar to inject cloud-scripts from working copy
The current build downloads its script from master by fetching a cloudstack
tarball. Besides being an unneeded load on the apache git server, this is a
problem when working on a branch and wanting to inject a different set of
scripts. It also makes it pretty likely that the injected copy of the script
will not match what a production release wants, so there is very little
chance of not needing to overwrite the scripts.

Ideally we would just rsync over some files. However, veewee does not provide
an option to do that. In order to keep a 'cleanly veewee-only' build possible,
and work with any recent veewee version, in this change we restor to using
shar (http://en.wikipedia.org/wiki/Shar) to produce an archive which can
execute as a script, which we feed to veewee to execute.
2014-09-22 21:38:14 +02:00
Leo Simons
35ba68444b CLOUDSTACK-7143: add the ability to inject a custom SSH key
When working on the systemvm in isolation, or using vagrant or similar tools,
it can be useful to inject a custom SSH key before merging a management server
systemvm.iso into it. This option allows that. It should _not_ have effect
on management-server-managed vms which always get their SSH keys injected.
2014-09-22 21:38:13 +02:00
Leo Simons
18364216d8 CLOUDSTACK-7143: improve build.sh failure handling and recovery
In particular, this refactoring allows the use of 'set -e' to exit early on
error. Previously the script would continue for a while when encountering a
problem, stuttering sometimes to (almost) completion, producing partial or
no results.

Added a bash on EXIT trap which runs add_on_exit cleanup code in the reverse
order it was specified. Resource cleanup is now co-located with resource
definition.

Added color-coded logging.

Made most of the hypervisor-specific exports optional. This script now
works on Mac OS X.
2014-09-22 21:38:13 +02:00
Leo Simons
2832bc47e1 CLOUDSTACK-7143: improve build.sh ergonomics
Added a bunch of possible arguments / environment variables, see the new usage
function for details (or run ./build.sh help).

Creates a new template with a unique name on every invocation, resulting in a
new virtualbox instance with a unique name. This makes it possible to build
multiple boxes at the same time, in parallel (including for example on a
multiuser box), and reduces the chance of failed build results conflicting
with each other.

Inject the version given on the command line (if any) into the created image.
2014-09-22 21:38:13 +02:00
Leo Simons
3c0cdb595f CLOUDSTACK-7143: merge systemvm templates, step 2: only one template remains 2014-09-22 21:31:33 +02:00
Rohit Yadav
4217d157c5 appliance: box commands for dumb bash in cleaning branch name, in build.sh
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit edb5650c737d582595e134aab286e7304f299983)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2014-08-08 21:24:18 +02:00
Rohit Yadav
e71009a571 appliance: fix branch finding method from e16414e even Junio can be wrong
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit bbaefb8e5def610a654fba907c8ec3f868416bdf)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2014-08-08 20:30:42 +02:00
Rohit Yadav
ffe0f2f60f appliance: use the right way to get git branch name
Taken from Junio C Hamano's blog [1], git's maintainer, he must not be wrong :)

[1] http://git-blame.blogspot.ch/2013/06/checking-current-branch-programatically.html

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit e16414e56d8f398f8ddbbde89d3ee833582b8bb2)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2014-08-08 19:22:20 +02:00
Rohit Yadav
a0006a6990 appliance: change the way we find git branch in build.sh
(cherry picked from commit 493c0a68d2332cb9275902128c28f926a909f4ed)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	tools/appliance/build.sh
2014-08-08 15:44:33 +02:00
Daan Hoogland
fec52be20e Revert "raw immage for ovm"
This reverts commit 23b1f228a1a779cbc5ba2f2f1f5e555684af6468.
2014-06-24 10:21:05 +02:00
Daan Hoogland
23b1f228a1 raw immage for ovm 2014-06-23 14:54:07 +02:00
Tim Mackey
a8212d9ef4 Cleanup of Xen and XenServer terms. Cloned xen plugin creating a xenserver plugin, then removed xen plugin
Signed-off-by: Tim Mackey <tmackey@gmail.com>
Signed-off-by: Sebastien Goasguen <runseb@gmail.com>
2014-06-07 04:50:23 -04:00
rayeesn
5f11c2e0be CLOUDSTACK-6392: system template always create with name master instead of specific branch
Signed-off-by: Abhinandan Prateek <aprateek@apache.org>
2014-05-28 11:35:57 +05:30
ynojima
570c3a2130 bugfix: systemvm template build failed
since newer vboxmanage generates $appliance-$branch-vmware-disk3.vmdk
2014-05-21 18:27:24 -06:00