29 Commits

Author SHA1 Message Date
Rohit Yadav
e7bd73e72b Merge branch '4.11' 2018-05-04 12:39:53 +05:30
Rohit Yadav
77172b9f03 vr: create tables before applying egress iptables rules
This fixes the issue that post-upgrade egress rules are not applied
on VR, restarting the network with cleanup used to be the workaround.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-05-04 12:37:23 +05:30
Rohit Yadav
8533def696 systemvm: Fixes #2561 patching on XenServer
This fixes incorrect xenstore-read binary path, this failed systemvm
to be patched/started correctly on xenserver. The other fix is to keep
the xen-domU flag that may be returned by virt-what. This effect
won't change the cmdline being consumed as the mgmt server side (java)
code sets the boot args in both xenstore and as pv args. The systemvm's
/boot is ext2 that can be booted by PyGrub on both old and recent
XenServer versions.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-05-04 12:37:23 +05:30
Rohit Yadav
ddc8d131c0 systemvmtemplate: Fixes #2541 adds Letsencrypt CA cert
On patching, the global cacerts keystore is imported in 'cloud' service
specific local keystore. This fixes #2541.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-05-04 12:37:23 +05:30
Rohit Yadav
4277b92abe Merge branch '4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-04-30 08:22:16 +02:00
Rohit Yadav
464551208c
xenserver: Add support for XS 7.3, 7.4 and XCP-ng 7.4 (#2605)
This adds support for XenServer 7.3 and 7.4, and XCP-ng 7.4 version as hypervisor hosts. Fixes #2523.

This also fixes the issue of 4.11 VRs stuck in starting for up-to 10mins, before they come up online.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-04-30 08:19:10 +02:00
Rohit Yadav
71ab3aff9a Merge branch '4.11' 2018-04-20 15:29:44 +05:30
Rohit Yadav
561630e449
router: Fix routing tables for public IP NAT based access (#2579)
This fixes routing table rule setup regression to correctly router
marked packets based on interface related ip route tables. This thereby
fixes the access of VMs in the same VPC using NAT/SNAT public IPs.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-04-20 15:29:04 +05:30
Rohit Yadav
644b0910cd Merge branch '4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-04-20 00:46:43 +05:30
Rafael Weingärtner
9288c64e5f systemvm: Use double quotes with 'RROUTER' variable in "common.sh" script (#2586)
While debugging the VR for #2579, I noticed that one of the scripts were breaking. The variable RROUTER was not set and this broke a conditional.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-04-20 00:27:45 +05:30
Rafael Weingärtner
15afc35ff9 Forward merge branch '4.11' (PR: #2576) to master 2018-04-18 13:11:44 -03:00
Rafael Weingärtner
bfe4cb0c41
Fix Python code checkstyle execute by "systemvm\test\runtests.sh" (#2576)
* dependencies update

* Add extra blank line required by ...!?

* fix W605 invalid escape sequence and more blank lines

* print all installed python packages versions
2018-04-18 13:07:37 -03:00
Rafael Weingärtner
20b93eaa06 Log command output in CsHelper.execute command (#2568) 2018-04-13 11:59:01 +02:00
Rohit Yadav
e71d4d4371 CLOUDSTACK-10304: turn off apache2 server tokens and signature in systemvms (#2563)
* systemvm: turn off apache2 server tokens and signature

This turns off apache2 server version signature/token in headers.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* systemvm: remove invalid code as conf.d is not available now

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-04-13 10:44:25 +02:00
Khosrow Moossavi
535e6153cc CLOUDSTACK-10232: SystemVMs and VR to run as HVM on XenServer (#2465)
Publishing boot args both to grub and xenstore-data and let
cloud-early-config decides if the VM is in PV or HVM mode
to read from correct source.
2018-03-27 15:48:37 +05:30
René Moser
c8dcc64b65 CLOUDSTACK-10341: VR minor fixes to systemvmtemplate (#2468)
- Fixes rsyslog: fix config error in rsylslog.conf

Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: action '*' treated as ':omusrmsg:*' - please use ':omusrmsg:*' syntax instead, '*' will not be supported in the future [v8.24.0 try http://www.rsyslog.com/e/2184 ]
Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: error during parsing file /etc/rsyslog.conf, on or before line 95: warnings occured in file '/etc/rsyslog.conf' around line 95 [v8.24.0 try http://www.rsyslog.com/e/2207 ]

- Run apache2 only after cloud-postinit

- Increase /run size for VR with 256M RAM

root@r-395-VM:~# systemctl daemon-reload
Failed to reload daemon: Refusing to reload, not enough space available on /run/systemd. Currently, 15.8M are free, but a safety buffer of 16.0M is enforced.

tmpfs            23M  6.5M   16M  29% /run
2018-03-23 11:52:29 +05:30
Rohit Yadav
ab0bce2a1b
CLOUDSTACK-10296: Find time different from last timestamp (#2458)
This fixes a difference issue in rVR heartbeat check script raised
recently on dev@.
Reduce logging to avoid logging to fill ramdisk
Make checkrouter return fault state when keepalived is not running

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-15 16:32:18 +05:30
Rohit Yadav
da8cf8c370
CLOUDSTACK-10319: Prefer TLSv1.2, deprecate TLSv1.0,1.1 (#2480)
This deprecates and remove TLS 1.0 and 1.1 from preferred list of
protocols and keeps only TLSv1.2.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-12 11:43:59 +01:00
Rohit Yadav
c0440e8124 CLOUDSTACK-10317: Fix SNAT rules for additional public nics (#2476)
* CLOUDSTACK-10317: Fix SNAT rules for additional public nics

This allows networks with additional public nics to have correct
SNAT iptables rules applied on configuration.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* update based on Wei's suggested change

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-03-08 10:01:36 +01:00
Marc-Aurèle Brothier
97441a82f9 CLOUDSTACK-10282: ipv6 firewall rules operation should be done with ip6tables (#2450)
For ipv6 firewall rules operation should be done with ip6tables.
2018-02-15 10:09:23 +01:00
Wido den Hollander
ce67726c6d CLOUDSTACK-10243: Do not use wait() on Python subprocess (#2421)
This might (and does block) in certain situations on the VR as
also explained in the Python documentation:

https://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait

  Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE
  and the child process generates enough output to a pipe such that
  it blocks waiting for the OS pipe buffer to accept more data.
  Use communicate() to avoid that.

Using the check_output function handles most of this for us and
also provides better error handling.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-02-10 18:27:00 +01:00
Rohit Yadav
61a5a29705
CLOUDSTACK-10252: Delete dnsmasq leases file on restart (#2427)
Delete dnsmasq's leases file when dnsmasq is restarted to avoid it
use old ip-mac-address-vm mapping leases.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-01-24 11:09:45 +01:00
Frank Maximus
3b23d5af74 CLOUDSTACK-10245: Fix password server regression (#2419)
In case of isolated, both self.config.is_vpc() and self.config.is_router() are false,
but self.config.is_dhcp() is true.
Moved the password server logic to the `if has_metadata` block,
as this is valid for all 3 systemvm types.
2018-01-23 17:20:03 +01:00
Frank Maximus
a9fdb31585 CLOUDSTACK-9749: Fix Password server running on internal LB VM (#2409)
Fixes code to start password server only on routers.
2018-01-19 13:41:57 +05:30
Wido den Hollander
e01dd89c93 CLOUDSTACK-10217: Clean up old MAC addresses from DHCP lease file (#2393)
When the IPv4 address of a Instance changes we need to make sure the
old entry is removed from the DHCP lease file on the Virtual Router
otherwise the Instance will still get the old lease.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-01-10 00:41:55 +05:30
Rohit Yadav
d19629a115 CLOUDSTACK-10013: Fixes based on code review and test failures
This includes test related fixes and code review fixes based on
reviews from @rafaelweingartner, @marcaurele, @wido and @DaanHoogland.

This also includes VMware disk-resize limitation bug fix based on comments
from @sateesh-chodapuneedi and @priyankparihar.

This also includes the final changes to systemvmtemplate and fixes to
code based on issues found via test failures.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 17:51:42 +05:30
Rohit Yadav
15b11a3b27 CLOUDSTACK-10013: Fix VMware related issues and fix misc tests
This fixes test failures around VMware with the new systemvmtemplate.
In addition:

- Does not skip rVR related test cases for VMware
- Removes rc.local
- Processes unprocessed cmd_line.json
- Fixed NPEs around VMware tests/code
- On VMware, use udevadm to reconfigure nic/mac address than rebooting
- Fix proper acpi shutdown script for faster systemvm shutdowns
- Give at least 256MB of swap for VRs to avoid OOM on VMware
- Fixes smoke tests for environment related failures

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 09:22:44 +05:30
Rohit Yadav
facc5945f0 CLOUDSTACK-10193: Fix smoke tests failures with new systemvmtemplate
- Several systemvmtemplate optimizations
- Uses new macchinina template for running smoke tests
- Switch to latest Debian 9.3.0 release for systemvmtemplate
- Introduce a new `get_test_template` that uses tiny test template
  such as macchinina as defined test_data.py
- rVR related fixes and improvements

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 09:22:44 +05:30
Rohit Yadav
85aee8d18d CLOUDSTACK-10013: SystemVM codebase refactorings and improvements
- Refactors and simplifies systemvm codebase file structures keeping
  the same resultant systemvm.iso packaging
- Password server systemd script and new postinit script that runs
  before sshd starts
- Fixes to keepalived and conntrackd config to make rVRs work again
- New /etc/issue featuring ascii based cloudmonkey logo/message and
  systemvmtemplate version
- SystemVM python codebase linted and tested. Added pylint/pep to
  Travis.
- iptables re-application fixes for non-VR systemvms.
- SystemVM template build fixes.
- Default secondary storage vm service offering boosted to have 2vCPUs
  and RAM equal to console proxy.
- Fixes to several marvin based smoke tests, especially rVR related
  tests. rVR tests to consider 3*advert_int+skew timeout before status
  is checked.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2017-12-23 09:22:44 +05:30