1889 Commits

Author SHA1 Message Date
Frank Maximus
b728e23e53 Marvin: Register template 2018-09-17 16:33:36 +02:00
Frank Maximus
604158220d CLOUDSTACK-10380: Refactoring of test_configdrive 2018-09-17 16:33:36 +02:00
Rohit Yadav
e559154a41 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-09-12 15:29:00 +05:30
Rohit Yadav
5a046e243a
systemvmtemplate: new 4.11.2 template and fixes (#2799)
VMware router will be rebooted based on #2794, per current config
the VRs on reboot will go through fsck checks slowing down the deployment
process by few seconds. This will ensure that fsck checks are done
on every 3rd boot of the VR. The `4` is used because 1st boot is done
during the build of systemvmtemplate appliance.

Add upgrade path for a new 4.11.2 systemvmtemplate.
Other changes:
- Add support for XS 7.5 Fixes #2834.
- Reboot VR only if mgmt gw is not pingable on vmware.
- Enable passive ftp by enabling nf_conntrack_helper. This is change in behaviour since linux 4.7

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-09-12 14:42:05 +05:30
Khosrow Moossavi
56f918583d Remove 'iam' projects (#2817)
* Remove 'iam' projects

* Drop corresponding tables
2018-09-11 13:06:20 -03:00
Rohit Yadav
494f708aea Merge remote-tracking branch 'origin/4.11' 2018-08-31 17:28:46 +05:30
Dennis Konrad
263559c19e systemvmtemplate: Bigger partition size and ambiguous redirect bugfix (#2832)
Fix partition sizes to allow the building of systemvmtemplate, and fix ambiguous redirect fix.
2018-08-31 17:17:51 +05:30
Rohit Yadav
75030e5522 Merge branch '4.11' 2018-08-01 16:37:21 +05:30
Khosrow Moossavi
2be51ac18d appliance: Fix typo in ISO url (#2775) 2018-07-31 13:04:37 +05:30
Rohit Yadav
c8a980b9cf Merge branch '4.11' 2018-07-26 13:08:41 +05:30
Khosrow Moossavi
52fb426231 systemvm: Update ISO URLs to the latest (#2747)
Update ISO URLs to the latest as the old URLs are broken. Base template updated for Debian 9.5.0.
2018-07-26 13:06:08 +05:30
Khosrow Moossavi
7c6630bca7 Cleanup POMs (#2613)
* Cleaup and code-formatting POM files

* Remove obsolete mycila license-maven-plugin

* Remove obsolete console-proxy/plugin project

* Move console-proxy-rdbconsole under console-proxy parent

* Use correct parent path for rdpconsole

* Order alphabetally items in setnextversion.sh

* Unifiy License header in POMs

* Alphabetic order of modules definition

* Extract all defined versions into parent pom

* Remove obsolete files: version-info.in, configure-info.in

* Remove redundant defaultGoal

* Remove useless checkstyle plugin from checkstyle project

* Order alphabetally items in pom.xml

* Add aditional SPACEs to fix debian build

* Don't execute checkstyle on parent projects

* Use UTF-8 encoding in building checkstyle project

* Extract plugin versions into properties

* Execute PMD plugin on all the projects with -Penablefindbugs

* Upgrade maven plugins to latest version

* Make sure to always look for apache parent pom from repository

* Fix incorrect version grep in debian packaging

* Fix rebase conflicts

* Fix rebase conflicts

* Remove PMD for now to be fixed on another PR
2018-07-25 14:39:37 -03:00
ernjvr
542d4da16c asyncjobs: add endtime to async jobs (#2739)
There is currently no functional mechanism that captures or persists the end time of when an asynchronous job has finished. As a result, users are not able to do any reporting about the duration of various asynchronous jobs in Cloudstack.
Link to FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Add+End+Time+To+Asynchronous+Jobs
2018-07-25 14:48:01 +05:30
Dingane Hlaluku
40af32b1b9 diagnostics: new diagnostics admin API for system VMs (#2721)
This is a new feature for CS that allows Admin users improved
troubleshooting of network issues in CloudStack hosted networks.

Description: For troubleshooting purposes, CloudStack administrators may wish to execute network utility commands remotely on system VMs, or request system VMs to ping/traceroute/arping to specific addresses over specific interfaces. An API command to provide such functionalities is being developed without altering any existing APIs. The targeted system VMs for this feature are the Virtual Router (VR), Secondary Storage VM (SSVM) and the Console Proxy VM (CPVM).

FS:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Remote+Diagnostics+API
ML discussion:
https://markmail.org/message/xt7owmb2c6iw7tva
2018-07-13 16:58:45 +05:30
Khosrow Moossavi
67860d9f46 maven: Updating pom.xml version numbers for release 4.11.2.0-SNAPSHOT (#2728)
Fixes the version in pom etc. to be consistent with versioning pattern as X.Y.Z.0-SNAPSHOT after a minor release.

Signed-off-by: Khosrow Moossavi <khos2ow@gmail.com>
2018-07-06 17:27:12 +05:30
Frank Maximus
89c2721fbd Fixes: #2725 2018-07-03 10:46:36 +02:00
Paul Angus
8ba318da19 Updating pom.xml version numbers for release 4.11.2-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-06-26 17:53:54 +01:00
Paul Angus
4ebafc44a9 Updating pom.xml version numbers for release 4.11.2-SNAPSHOT
Signed-off-by: Paul Angus <paul.angus@shapeblue.com>
2018-06-26 17:24:36 +01:00
Paul Angus
2cb2dacbe7 Updating pom.xml version numbers for release 4.11.1.0
Signed-off-by: Paul Angus <paulangus@PA-Ansible-GUI.sblab.local>
2018-06-21 15:52:43 +01:00
Rohit Yadav
85750f918b Merge branch '4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-06-20 12:31:52 +05:30
Rohit Yadav
c87ca1b262 packaging: use libuuid x86_64 package for cloudstack-common (#2706)
* packaging: use libuuid x86_64 package for cloudstack-common

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* 64 bit links is packaged

* post scan filter to exclude libuuid.so.1

* Revert "packaging: use libuuid x86_64 package for cloudstack-common"

This reverts commit b3fb8957fe4e98c85949be2010f0316c89d535a9.

* post scan filter to exclude libuuid.so.1 (centos63)

* revert removal of 32 bit support for vhd-util libs
2018-06-19 13:34:44 +02:00
Rohit Yadav
e7bd73e72b Merge branch '4.11' 2018-05-04 12:39:53 +05:30
Rohit Yadav
8533def696 systemvm: Fixes #2561 patching on XenServer
This fixes incorrect xenstore-read binary path, this failed systemvm
to be patched/started correctly on xenserver. The other fix is to keep
the xen-domU flag that may be returned by virt-what. This effect
won't change the cmdline being consumed as the mgmt server side (java)
code sets the boot args in both xenstore and as pv args. The systemvm's
/boot is ext2 that can be booted by PyGrub on both old and recent
XenServer versions.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-05-04 12:37:23 +05:30
Rohit Yadav
ddc8d131c0 systemvmtemplate: Fixes #2541 adds Letsencrypt CA cert
On patching, the global cacerts keystore is imported in 'cloud' service
specific local keystore. This fixes #2541.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-05-04 12:37:23 +05:30
Marc-Aurèle Brothier
46bd94c6a2 [CLOUDSTACK-10254] checkstyle: add package name declaration validation (#2422)
* checktyle: verify package name matches directory structure

* fix new checkstyle findings on directory with package name mismatch
2018-04-26 10:32:08 -03:00
Rafael Weingärtner
b3c22df71d Forward merge branch '4.11' to master
ConfigDrive fixes: CLOUDSTACK-10288, CLOUDSTACK-10289 (#2566)
CLOUDSTACK-9677: Adding storage policy support for swift as secondary
storage (#2412)
2018-04-26 10:14:49 -03:00
Frank Maximus
85203248a4 ConfigDrive fixes: CLOUDSTACK-10288, CLOUDSTACK-10289 (#2566)
* CLOUDSTACK-10289: Config Drive Metadata: Use VM UUID instead of VM id

* CLOUDSTACK-10288: Config Drive Userdata: support for binary userdata

* CLOUDSTACK-10358: SSH keys are missing on Config Drive disk in some cases
2018-04-26 10:10:23 -03:00
Rafael Weingärtner
15afc35ff9 Forward merge branch '4.11' (PR: #2576) to master 2018-04-18 13:11:44 -03:00
Rafael Weingärtner
bfe4cb0c41
Fix Python code checkstyle execute by "systemvm\test\runtests.sh" (#2576)
* dependencies update

* Add extra blank line required by ...!?

* fix W605 invalid escape sequence and more blank lines

* print all installed python packages versions
2018-04-18 13:07:37 -03:00
Rohit Yadav
ba61b512b2 [Merge 4.11] CLOUDSTACK-10341: Reduce systemvmtemplate size (#2506)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-28 13:21:59 +05:30
Rohit Yadav
986ecfa7e9
CLOUDSTACK-10341: Reduce systemvmtemplate size, install nftables (#2506)
This reduces systemvmtemplate size by 600MB and installs nftables,
updates iptables. This also fixes a failing smoke test.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-28 13:11:51 +05:30
Rohit Yadav
9753cc3681 Merge branch '4.11'
CLOUDSTACK-10341: VR minor fixes to systemvmtemplate (#2468)
CLOUDSTACK-10340: Add setter to hypervisorType in VMInstanceVO (#2504)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-23 12:00:20 +05:30
René Moser
c8dcc64b65 CLOUDSTACK-10341: VR minor fixes to systemvmtemplate (#2468)
- Fixes rsyslog: fix config error in rsylslog.conf

Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: action '*' treated as ':omusrmsg:*' - please use ':omusrmsg:*' syntax instead, '*' will not be supported in the future [v8.24.0 try http://www.rsyslog.com/e/2184 ]
Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: error during parsing file /etc/rsyslog.conf, on or before line 95: warnings occured in file '/etc/rsyslog.conf' around line 95 [v8.24.0 try http://www.rsyslog.com/e/2207 ]

- Run apache2 only after cloud-postinit

- Increase /run size for VR with 256M RAM

root@r-395-VM:~# systemctl daemon-reload
Failed to reload daemon: Refusing to reload, not enough space available on /run/systemd. Currently, 15.8M are free, but a safety buffer of 16.0M is enforced.

tmpfs            23M  6.5M   16M  29% /run
2018-03-23 11:52:29 +05:30
Rohit Yadav
8ef131745a Merge branch '4.11' 2018-03-15 16:46:50 +05:30
Khosrow Moossavi
1708838518 CLOUDSTACK-10268: Fix and enhance package script (#2433)
- new flag `-T, --use-timestamp` to use `timestamp` when POM version contains SNAPSHOT
  - in the final artifacts (jar) name
  - in the final package (rpm, deb) name
  - in `/etc/cloudstack-release` file of SystemVMs
  - in the Management Server > About dialog
- if there's a "branding" string in the POM version (e.g. `x.y.z.a-NAME[-SNAPSHOT]`),
the branding name will be used in the final generated pacakge name such as following:
  - `cloudstack-management-x.y.z.a-NAME.NUMBER.el7.centos.x86_64`
  - `cloudstack-management_x.y.z.a-NAME-NUMBER~xenial_all.deb`
- branding string can be overriden with newly added `-b, --brand` flag
- handle the new format version for VR version
- fix long opts (they were broken)
- tolerate and show a warning message for unrecognized flags
- usage help reformat

* Deprecate Version class in favor or CloudStackVersion
2018-03-15 10:55:29 +00:00
Raf Smeets
19d6578732 CLOUDSTACK-10303 : Refactor test data to nuage_test_data.py runnable against simulator (#2483)
* Refactored nuage tests

Added simulator support for ConfigDrive
Allow all nuage tests to run against simulator
Refactored nuage tests to remove code duplication

* Move test data from test_data.py to nuage_test_data.py

Nuage test data is now contained in nuage_test_data.py instead of
test_data.py
Removed all nuage test data from nuage_test_data.py

* CLOUD-1252 fixed cleanup of vpc tier network

* Import libVSD into the codebase

* CLOUDSTACK-1253: Volumes are not expunged in simulator

* Fixed some merge issues in test_nuage_vsp_mngd_subnets test

* Implement GetVolumeStatsCommand in Simulator

* Add vspk as marvin nuagevsp dependency, after removing libVSD dependency

* correct libVSD files for license purposes

pep8 pyflakes compliant
2018-03-14 17:17:36 +05:30
Rohit Yadav
989e4e29dc Merge branch '4.11' 2018-03-14 15:14:59 +05:30
Khosrow Moossavi
7112affe19 Bump the version of Debian net-installer to 9.4.0 (#2485) 2018-03-13 09:46:42 +01:00
Rohit Yadav
82bcc74679 Merge branch '4.11' 2018-03-12 16:15:46 +05:30
Rohit Yadav
da8cf8c370
CLOUDSTACK-10319: Prefer TLSv1.2, deprecate TLSv1.0,1.1 (#2480)
This deprecates and remove TLS 1.0 and 1.1 from preferred list of
protocols and keeps only TLSv1.2.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-12 11:43:59 +01:00
Daan Hoogland
cdddecb41e Merge release branch 4.11 to master
* 4.11:
  CLOUDSTACK-10306: Upgrade to VMware 6.5 vim jar dependency (#2467)
  CLOUDSTACK-10298: fix for recreation of an earlier deleted Nuage managed network (#2460)
2018-02-27 12:42:41 +01:00
Sigert Goeminne
3dfbcbc48a CLOUDSTACK-10298: fix for recreation of an earlier deleted Nuage managed network (#2460)
* CLOUDSTACK-10298: Recreation of an earlier deleted Nuage managed isolated or vpc tier network fails

Added negative tests for feature vsd managed subnets
Added fixes for reacreation of an earlier deleted Nuage managed network
Updated Nuage Vsp CloudStack client to 1.0.8
Co-authored by: Sigert Goeminne <sigert.goeminne@nuagenetworks.net>

* removed commented code as per review comment
2018-02-27 12:25:15 +01:00
Rohit Yadav
0ece15f86e Updating pom.xml version numbers for release 4.11.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-02-26 16:57:48 +01:00
Dmytro Shevchenko
eeba3e4853 CLOUDSTACK-10295 Marvin: add support for password-enabled templates 2018-02-15 21:06:17 +02:00
Rohit Yadav
6ffbce6159 Updating pom.xml version numbers for release 4.11.0.1-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-02-05 11:13:50 +01:00
Rohit Yadav
5dada1f7ed Updating pom.xml version numbers for release 4.11.0.0
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-01-26 13:13:37 +01:00
Marc-Aurèle Brothier
893a88d225 CLOUDSTACK-10105: Use maven standard project structure in all projects (#2283)
Remove maven standard module (which only a few were using) and get ride of maven customization for the projects structure.

- moved all directories to src/main/java, src/main/resources, src/main/scripts, src/test/java, src/test/resources
- grep scan to search for src/com and src/org left over
- grep for <project>/scripts to fix pom.xml configuration
- remove custom <build> configuration in pom.xml

Signed-off-by: Marc-Aurèle Brothier <m@brothier.org>
2018-01-20 03:19:27 +05:30
Rohit Yadav
072dbc0720 Updating pom.xml version numbers for master to 4.12.0.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-01-15 17:43:45 +05:30
Mike Tutkowski
a30a31c9b7 CLOUDSTACK-9620: Enhancements for managed storage (#2298)
Allowed zone-wide primary storage based on a custom plug-in to be added via the GUI in a KVM-only environment (previously this only worked for XenServer and VMware)

Added support for root disks on managed storage with KVM

Added support for volume snapshots with managed storage on KVM

Enable creating a template directly from a volume (i.e. without having to go through a volume snapshot) on KVM with managed storage

Only allow the resizing of a volume for managed storage on KVM if the volume in question is either not attached to a VM or is attached to a VM in the Stopped state.

Included support for Reinstall VM on KVM with managed storage

Enabled offline migration on KVM from non-managed storage to managed storage and vice versa

Included support for online storage migration on KVM with managed storage (NFS and Ceph to managed storage)

Added support to download (extract) a managed-storage volume to a QCOW2 file

When uploading a file from outside of CloudStack to CloudStack, set the min and max IOPS, if applicable.

Included support for the KVM auto-convergence feature

The compression flag was actually added in version 1.0.3 (1000003) as opposed to version 1.3.0 (1003000) (changed this to reflect the correct version)

On KVM when using iSCSI-based managed storage, if the user shuts a VM down from the guest OS (as opposed to doing so from CloudStack), we need to pass to the KVM agent a list of applicable iSCSI volumes that need to be disconnected.

Added a new Global Setting: kvm.storage.live.migration.wait

For XenServer, added a check to enforce that only volumes from zone-wide managed storage can be storage motioned from a host in one cluster to a host in another cluster (cannot do so at the time being with volumes from cluster-scoped managed storage)

Don’t allow Storage XenMotion on a VM that has any managed-storage volume with one or more snapshots.

Enabled for managed storage with VMware: Template caching, create snapshot, delete snapshot, create volume from snapshot, and create template from snapshot

Added an SIOC API plug-in to support VMware SIOC

When starting a VM that uses managed storage in a cluster other than the one it last was running in, we need to remove the reference to the iSCSI volume from the original cluster.

Added the ability to revert a volume to a snapshot

Enabled cluster-scoped managed storage

Added support for VMware dynamic discovery
2018-01-15 00:05:52 +05:30
Frank Maximus
b176648f90 CLOUDSTACK-9813: Extending Config Drive support (#2097)
Extending Config Drive support

* Added support for VMware
* Build configdrive.iso on ssvm
* Added support for VPC and Isolated Networks
* Moved implementation to new Service Provider
* UI fix: add support for urlencoded userdata
* Add support for building systemvm behind a proxy

Co-Authored-By: Raf Smeets <raf.smeets@nuagenetworks.net>
Co-Authored-By: Frank Maximus <frank.maximus@nuagenetworks.net>
Co-Authored-By: Sigert Goeminne <sigert.goeminne@nuagenetworks.net>
2018-01-12 15:14:40 +05:30