31322 Commits

Author SHA1 Message Date
Rohit Yadav
8f881ce08b [Merge 4.11] CLOUDSTACK-10232: SystemVMs and VR to run as HVM on XenServer (#2465)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-27 17:12:26 +05:30
Rafael Weingärtner
0afcec60f4 [CLOUDSTACK-10307] Remove unused things from HostDaoImpl (#2438)
* Remove unused code from CloudZonesStartupProcessor

* Removed unused @Component annotation from HostDaoImpl And unused method
2018-03-27 10:46:25 +00:00
Khosrow Moossavi
535e6153cc CLOUDSTACK-10232: SystemVMs and VR to run as HVM on XenServer (#2465)
Publishing boot args both to grub and xenstore-data and let
cloud-early-config decides if the VM is in PV or HVM mode
to read from correct source.
2018-03-27 15:48:37 +05:30
Rohit Yadav
9222da2d62 [Merge 4.11] CLOUDSTACK-10319: Allow TLSv1, v1.1 for XenServer, Vmware (#2507)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-26 14:17:57 +05:30
Rohit Yadav
c4cc679c3b
CLOUDSTACK-10319: Allow TLSv1, v1.1 for XenServer, Vmware (#2507)
This reverts changes from #2480, instead moves TLS settings to
java ciphers settings config file. It should be sufficient to enforce
TLS v1.2 on public facing CloudStack services:
- CloudStack webserver (Jetty based)
- Apache2 for secondary storage VM
- CPVM HTTPs server

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-26 14:16:49 +05:30
Wido den Hollander
9733a10ecd CLOUDSTACK-10221: Allow IPv6 when creating a Basic Network (#2397)
Since CloudStack 4.10 Basic Networking supports IPv6 and thus
should be allowed to be specified when creating a network.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-03-26 07:36:57 +00:00
Marc-Aurèle Brothier
ca1760a46b CLOUDSTACK-10320 - Invalid pair for response object breaking response parsing (#2481) 2018-03-23 16:55:22 +00:00
Rohit Yadav
9753cc3681 Merge branch '4.11'
CLOUDSTACK-10341: VR minor fixes to systemvmtemplate (#2468)
CLOUDSTACK-10340: Add setter to hypervisorType in VMInstanceVO (#2504)

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-23 12:00:20 +05:30
René Moser
c8dcc64b65 CLOUDSTACK-10341: VR minor fixes to systemvmtemplate (#2468)
- Fixes rsyslog: fix config error in rsylslog.conf

Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: action '*' treated as ':omusrmsg:*' - please use ':omusrmsg:*' syntax instead, '*' will not be supported in the future [v8.24.0 try http://www.rsyslog.com/e/2184 ]
Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: error during parsing file /etc/rsyslog.conf, on or before line 95: warnings occured in file '/etc/rsyslog.conf' around line 95 [v8.24.0 try http://www.rsyslog.com/e/2207 ]

- Run apache2 only after cloud-postinit

- Increase /run size for VR with 256M RAM

root@r-395-VM:~# systemctl daemon-reload
Failed to reload daemon: Refusing to reload, not enough space available on /run/systemd. Currently, 15.8M are free, but a safety buffer of 16.0M is enforced.

tmpfs            23M  6.5M   16M  29% /run
2018-03-23 11:52:29 +05:30
Rohit Yadav
2a068696f8
CLOUDSTACK-10340: Add setter to hypervisorType in VMInstanceVO (#2504)
This adds a missing setter to set hypervisorType in VMInstanceVO.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-23 11:48:13 +05:30
nvazquez
1c99fd7388 Merge branch '4.11' 2018-03-21 08:12:59 -03:00
Olivier Lemasle
3e62ce9c1b CLOUDSTACK-10331: Remove reference to deleted script vm_snapshots.js (#2497) 2018-03-21 08:00:17 -03:00
Nicolas Vazquez
6a75423779 CLOUDSTACK-10231: Asserted fixes for Direct Download on KVM (#2408)
Several fixes addressed:

- Dettach ISO fails when trying to detach a direct download ISO
- Fix for metalink support on SSVM agents (this closes CLOUDSTACK-10238)
- Reinstall VM from bypassed registered template (this closes CLOUDSTACK-10250)
- Fix upload certificate error message even though operation was successful
- Fix metalink download, checksum retry logic and metalink SSVM downloader
2018-03-20 19:24:46 +05:30
Rafael Weingärtner
cd3a128090
[CLOUDSTACK-10329] Button in ACL rules page to export all rules as a CSV file (#2494) 2018-03-20 07:36:24 -03:00
Khosrow Moossavi
af52b1a873 CLOUDSTACK-10268: Fix typo (#2495)
This fixes typo introduced in PR #2433
2018-03-19 11:46:29 +05:30
Rohit Yadav
9645d9c34c Merge branch '4.11' 2018-03-19 11:42:22 +05:30
Rohit Yadav
1ad04cbc9b
CLOUDSTACK-10330: Add PULL_REQUEST_TEMPLATE (#2489)
This adds a standard pull request template.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-19 11:40:57 +05:30
Rafael Weingärtner
8a3943b763 CLOUDSTACK-10132: Fix POM of "agent-lb" project (#2491)
Post fwd merging, this fixes build issues.
2018-03-16 08:04:22 +05:30
Rohit Yadav
8ef131745a Merge branch '4.11' 2018-03-15 16:46:50 +05:30
Rohit Yadav
30175d6879
CLOUDSTACK-10132: Extend support for management servers LB for agents (#2469)
The new CA framework introduced basic support for comma-separated
list of management servers for agent, which makes an external LB
unnecessary.

This extends that feature to implement LB sorting algorithms that
sorts the management server list before they are sent to the agents.
This adds a central intelligence in the management server and adds
additional enhancements to Agent class to be algorithm aware and
have a background mechanism to check/fallback to preferred management
server (assumed as the first in the list). This is support for any
indirect agent such as the KVM, CPVM and SSVM agent, and would
provide support for management server host migration during upgrade
(when instead of in-place, new hosts are used to setup new mgmt server).

This FR introduces two new global settings:

- `indirect.agent.lb.algorithm`: The algorithm for the indirect agent LB.
- `indirect.agent.lb.check.interval`: The preferred host check interval
  for the agent's background task that checks and switches to agent's
  preferred host.

The indirect.agent.lb.algorithm supports following algorithm options:

- static: use the list as provided.
- roundrobin: evenly spreads hosts across management servers based on
  host's id.
- shuffle: (pseudo) randomly sorts the list (not recommended for production).

Any changes to the global settings - `indirect.agent.lb.algorithm` and
`host` does not require restarting of the mangement server(s) and the
agents. A message bus based system dynamically reacts to change in these
global settings and propagates them to all connected agents.

Comma-separated management server list is propagated to agents on
following cases:
- Addition of a host (including ssvm, cpvm systevms).
- Connection or reconnection by the agents to a management server.
- After admin changes the 'host' and/or the
  'indirect.agent.lb.algorithm' global settings.

On the agent side, the 'host' setting is saved in its properties file as:
`host=<comma separated addresses>@<algorithm name>`.

First the agent connects to the management server and sends its current
management server list, which is compared by the management server and
in case of failure a new/update list is sent for the agent to persist.

From the agent's perspective, the first address in the propagated list
will be considered the preferred host. A new background task can be
activated by configuring the `indirect.agent.lb.check.interval` which is
a cluster level global setting from CloudStack and admins can also
override this by configuring the 'host.lb.check.interval' in the
`agent.properties` file.

Every time agent gets a ms-host list and the algorithm, the host specific
background check interval is also sent and it dynamically reconfigures
the background task without need to restart agents.

Note: The 'static' and 'roundrobin' algorithms, strictly checks for the
order as expected by them, however, the 'shuffle' algorithm just checks
for content and not the order of the comma separate ms host addresses.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-15 16:34:03 +05:30
Rohit Yadav
ab0bce2a1b
CLOUDSTACK-10296: Find time different from last timestamp (#2458)
This fixes a difference issue in rVR heartbeat check script raised
recently on dev@.
Reduce logging to avoid logging to fill ramdisk
Make checkrouter return fault state when keepalived is not running

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-15 16:32:18 +05:30
Khosrow Moossavi
1708838518 CLOUDSTACK-10268: Fix and enhance package script (#2433)
- new flag `-T, --use-timestamp` to use `timestamp` when POM version contains SNAPSHOT
  - in the final artifacts (jar) name
  - in the final package (rpm, deb) name
  - in `/etc/cloudstack-release` file of SystemVMs
  - in the Management Server > About dialog
- if there's a "branding" string in the POM version (e.g. `x.y.z.a-NAME[-SNAPSHOT]`),
the branding name will be used in the final generated pacakge name such as following:
  - `cloudstack-management-x.y.z.a-NAME.NUMBER.el7.centos.x86_64`
  - `cloudstack-management_x.y.z.a-NAME-NUMBER~xenial_all.deb`
- branding string can be overriden with newly added `-b, --brand` flag
- handle the new format version for VR version
- fix long opts (they were broken)
- tolerate and show a warning message for unrecognized flags
- usage help reformat

* Deprecate Version class in favor or CloudStackVersion
2018-03-15 10:55:29 +00:00
Rafael Weingärtner
972b8b71d7
CLOUDSTACK-8855 Improve Error Message for Host Alert State and reconnect host API. (#2387)
* CLOUDSTACK-8855 Improve Error Message for Host Alert State

* [CLOUDSTACK-9846] create column to save the content of alert messages

Remove declaration of throws CloudRuntimeException
I also removed some unused variables and comments left behind

This closes #837

* Isolate a problematic test "smoke/test_certauthority_root"
2018-03-14 15:27:43 -03:00
Nicolas Vazquez
74db647dbb CLOUDSTACK-10321: CPU Cap for KVM (#2482) 2018-03-14 18:21:24 +00:00
Raf Smeets
19d6578732 CLOUDSTACK-10303 : Refactor test data to nuage_test_data.py runnable against simulator (#2483)
* Refactored nuage tests

Added simulator support for ConfigDrive
Allow all nuage tests to run against simulator
Refactored nuage tests to remove code duplication

* Move test data from test_data.py to nuage_test_data.py

Nuage test data is now contained in nuage_test_data.py instead of
test_data.py
Removed all nuage test data from nuage_test_data.py

* CLOUD-1252 fixed cleanup of vpc tier network

* Import libVSD into the codebase

* CLOUDSTACK-1253: Volumes are not expunged in simulator

* Fixed some merge issues in test_nuage_vsp_mngd_subnets test

* Implement GetVolumeStatsCommand in Simulator

* Add vspk as marvin nuagevsp dependency, after removing libVSD dependency

* correct libVSD files for license purposes

pep8 pyflakes compliant
2018-03-14 17:17:36 +05:30
Rohit Yadav
989e4e29dc Merge branch '4.11' 2018-03-14 15:14:59 +05:30
Henko
c3488a51db CLOUDSTACK-10147 Disabled Xenserver Cluster can still deploy VM's. Added code to skip disabled clusters when selecting a host (#2442) 2018-03-13 13:27:51 -03:00
Yoan Blanc
521e71fbeb createNetworkACL: number has the wrong doc (#2484)
Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-03-13 11:42:42 -03:00
Rafael Weingärtner
7efdaa65f7
[CLOUDSTACK-10314] Add Text-Field to each ACL Rule (#2475)
* [CLOUDSTACK-10314] Add Text-Field to each ACL Rule

It is interesting to have a text field (e.g. CHAR-256) added to each ACL rule, which allows to enter a "reason" for each FW Rule created. This is valuable for customer documentation, as well as best practice for an evidence towards auditing the system

* Formatting to make check style happy and code clean ups
2018-03-13 11:07:35 -03:00
Khosrow Moossavi
7112affe19 Bump the version of Debian net-installer to 9.4.0 (#2485) 2018-03-13 09:46:42 +01:00
Rohit Yadav
82bcc74679 Merge branch '4.11' 2018-03-12 16:15:46 +05:30
Rohit Yadav
da8cf8c370
CLOUDSTACK-10319: Prefer TLSv1.2, deprecate TLSv1.0,1.1 (#2480)
This deprecates and remove TLS 1.0 and 1.1 from preferred list of
protocols and keeps only TLSv1.2.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-12 11:43:59 +01:00
Rohit Yadav
68251e8440 Merge branch '4.11' 2018-03-08 12:19:59 +01:00
Rafael Weingärtner
c3e5c09ac6
[CLOUDSTACK-10197] Update DisplayText of XenServer tools ISO entry in the database when it already exists (#2470)
* Update the displayText of XenServer ISO when it already exist in the DB

Besides updating the ISO display text, I also created unit test cases for 'createXenServerToolsIsoEntryInDatabase' and 'getActualIsoTemplate' methods.

* Formatting and cleanups for checkstyle of changed classes
2018-03-08 06:59:52 -03:00
Rohit Yadav
c0440e8124 CLOUDSTACK-10317: Fix SNAT rules for additional public nics (#2476)
* CLOUDSTACK-10317: Fix SNAT rules for additional public nics

This allows networks with additional public nics to have correct
SNAT iptables rules applied on configuration.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* update based on Wei's suggested change

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-08 10:01:36 +01:00
Rafael Weingärtner
f2efbcecec
[CLOUDSTACK-10240] ACS cannot migrate a local volume to shared storage (#2425)
* [CLOUDSTACK-10240] ACS cannot migrate a volume from local to shared storage.

CloudStack is logically restricting the migration of local storages to shared storage and vice versa. This restriction is a logical one and can be removed for XenServer deployments. Therefore, we will enable migration of volumes between local-shared storages in XenServers independently of their service offering. This will work as an override mechanism to the disk offering used by volumes. If administrators want to migrate local volumes to a shared storage, they should be able to do so (the hypervisor already allows that). The same the other way around.

* Cleanups implemented while working on [CLOUDSTACK-10240]

* Fix test case test_03_migrate_options_storage_tags

The changes applied were:
- When loading hypervisors capabilities we must use "default" instead of nulls
- "Enable" storage migration for simulator hypervisor
- Remove restriction on "ClusterScopeStoragePoolAllocator" to find shared pools
2018-03-07 18:23:15 -03:00
Nicolas Vazquez
c0b920f740 CLOUDSTACK-10274: L2 network refused to be designed on VXLAN physical network (#2448)
L2 network refused to be designed on VXLAN physical network. Add fix for vxlan issue.
Add condition for L2 networks which do not allow specifying vlan.
2018-03-06 14:56:47 +01:00
Rafael Weingärtner
4412563f19
[CLOUDSTACK-10318] Bug on sorting ACL rules list in chrome (#2478) 2018-03-06 07:53:10 -03:00
Rafael Weingärtner
2037dc9eb3
[CLOUDSTACK-10257]Create template/volume does not allow to specify HVM requirement (#2437) 2018-03-02 08:07:53 -03:00
Rafael Weingärtner
9aa34c4092
Merge pull request #2439 from rafaelweingartner/CLOUDSTACK-10259
[CLOUDSTACK-10259] Missing float part of secondary storage data in listAccounts method
2018-03-01 13:34:37 -03:00
Yoan Blanc
2ad5202823 dateutil: constistency of tzdate input and output (#2392)
Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
Signed-off-by: Daan Hoogland <daan.hoogland@shapeblue.com>
2018-02-28 12:28:10 +01:00
Rafael Weingärtner
5a5b1354bd
Merge pull request #2463 from rafaelweingartner/createPathUpgrade4.12.0.0
[CLOUDSTACK-10302] Create database path upgrade from 4.11.0.0 to 4.12.0.0
2018-02-27 16:27:35 -03:00
Rafael Weingärtner
e1b1246993 Forward merge #2464 merged on '4.11' branch
[4.11] CLOUDSTACK-10299: UI: fix error in network listing in project mode
2018-02-27 16:09:12 +01:00
Rafael Weingärtner
0bb20a7ed2
Merge pull request #2464 from resmo/fix/4.11/CLOUDSTACK-10299
[4.11] CLOUDSTACK-10299: UI: fix error in network listing in project mode
2018-02-27 12:04:57 -03:00
mrunalinikankariya
bb607d07a9 CLOUDSTACK-10054:Volume download times out in 3600 seconds (#2244) 2018-02-27 13:33:00 +01:00
Daan Hoogland
cdddecb41e Merge release branch 4.11 to master
* 4.11:
  CLOUDSTACK-10306: Upgrade to VMware 6.5 vim jar dependency (#2467)
  CLOUDSTACK-10298: fix for recreation of an earlier deleted Nuage managed network (#2460)
2018-02-27 12:42:41 +01:00
Rohit Yadav
5c2b8f80f0 CLOUDSTACK-10306: Upgrade to VMware 6.5 vim jar dependency (#2467)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-02-27 12:28:48 +01:00
Sigert Goeminne
3dfbcbc48a CLOUDSTACK-10298: fix for recreation of an earlier deleted Nuage managed network (#2460)
* CLOUDSTACK-10298: Recreation of an earlier deleted Nuage managed isolated or vpc tier network fails

Added negative tests for feature vsd managed subnets
Added fixes for reacreation of an earlier deleted Nuage managed network
Updated Nuage Vsp CloudStack client to 1.0.8
Co-authored by: Sigert Goeminne <sigert.goeminne@nuagenetworks.net>

* removed commented code as per review comment
2018-02-27 12:25:15 +01:00
Rohit Yadav
026eab0035 Merge branch '4.11' 2018-02-26 16:59:22 +01:00
Rohit Yadav
0ece15f86e Updating pom.xml version numbers for release 4.11.1.0-SNAPSHOT
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-02-26 16:57:48 +01:00