This adds support for XenServer 7.3 and 7.4, and XCP-ng 7.4 version as hypervisor hosts. Fixes#2523.
This also fixes the issue of 4.11 VRs stuck in starting for up-to 10mins, before they come up online.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This fixes routing table rule setup regression to correctly router
marked packets based on interface related ip route tables. This thereby
fixes the access of VMs in the same VPC using NAT/SNAT public IPs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
While debugging the VR for #2579, I noticed that one of the scripts were breaking. The variable RROUTER was not set and this broke a conditional.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* dependencies update
* Add extra blank line required by ...!?
* fix W605 invalid escape sequence and more blank lines
* print all installed python packages versions
* systemvm: turn off apache2 server tokens and signature
This turns off apache2 server version signature/token in headers.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* systemvm: remove invalid code as conf.d is not available now
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
CLOUDSTACK-10341: VR minor fixes to systemvmtemplate (#2468)
CLOUDSTACK-10340: Add setter to hypervisorType in VMInstanceVO (#2504)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Fixes rsyslog: fix config error in rsylslog.conf
Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: action '*' treated as ':omusrmsg:*' - please use ':omusrmsg:*' syntax instead, '*' will not be supported in the future [v8.24.0 try http://www.rsyslog.com/e/2184 ]
Feb 26 08:09:54 r-413-VM liblogging-stdlog[19754]: error during parsing file /etc/rsyslog.conf, on or before line 95: warnings occured in file '/etc/rsyslog.conf' around line 95 [v8.24.0 try http://www.rsyslog.com/e/2207 ]
- Run apache2 only after cloud-postinit
- Increase /run size for VR with 256M RAM
root@r-395-VM:~# systemctl daemon-reload
Failed to reload daemon: Refusing to reload, not enough space available on /run/systemd. Currently, 15.8M are free, but a safety buffer of 16.0M is enforced.
tmpfs 23M 6.5M 16M 29% /run
This fixes a difference issue in rVR heartbeat check script raised
recently on dev@.
Reduce logging to avoid logging to fill ramdisk
Make checkrouter return fault state when keepalived is not running
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This deprecates and remove TLS 1.0 and 1.1 from preferred list of
protocols and keeps only TLSv1.2.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* CLOUDSTACK-10317: Fix SNAT rules for additional public nics
This allows networks with additional public nics to have correct
SNAT iptables rules applied on configuration.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* update based on Wei's suggested change
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This might (and does block) in certain situations on the VR as
also explained in the Python documentation:
https://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait
Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE
and the child process generates enough output to a pipe such that
it blocks waiting for the OS pipe buffer to accept more data.
Use communicate() to avoid that.
Using the check_output function handles most of this for us and
also provides better error handling.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Delete dnsmasq's leases file when dnsmasq is restarted to avoid it
use old ip-mac-address-vm mapping leases.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In case of isolated, both self.config.is_vpc() and self.config.is_router() are false,
but self.config.is_dhcp() is true.
Moved the password server logic to the `if has_metadata` block,
as this is valid for all 3 systemvm types.
When the IPv4 address of a Instance changes we need to make sure the
old entry is removed from the DHCP lease file on the Virtual Router
otherwise the Instance will still get the old lease.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
This includes test related fixes and code review fixes based on
reviews from @rafaelweingartner, @marcaurele, @wido and @DaanHoogland.
This also includes VMware disk-resize limitation bug fix based on comments
from @sateesh-chodapuneedi and @priyankparihar.
This also includes the final changes to systemvmtemplate and fixes to
code based on issues found via test failures.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This fixes test failures around VMware with the new systemvmtemplate.
In addition:
- Does not skip rVR related test cases for VMware
- Removes rc.local
- Processes unprocessed cmd_line.json
- Fixed NPEs around VMware tests/code
- On VMware, use udevadm to reconfigure nic/mac address than rebooting
- Fix proper acpi shutdown script for faster systemvm shutdowns
- Give at least 256MB of swap for VRs to avoid OOM on VMware
- Fixes smoke tests for environment related failures
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Several systemvmtemplate optimizations
- Uses new macchinina template for running smoke tests
- Switch to latest Debian 9.3.0 release for systemvmtemplate
- Introduce a new `get_test_template` that uses tiny test template
such as macchinina as defined test_data.py
- rVR related fixes and improvements
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Refactors and simplifies systemvm codebase file structures keeping
the same resultant systemvm.iso packaging
- Password server systemd script and new postinit script that runs
before sshd starts
- Fixes to keepalived and conntrackd config to make rVRs work again
- New /etc/issue featuring ascii based cloudmonkey logo/message and
systemvmtemplate version
- SystemVM python codebase linted and tested. Added pylint/pep to
Travis.
- iptables re-application fixes for non-VR systemvms.
- SystemVM template build fixes.
- Default secondary storage vm service offering boosted to have 2vCPUs
and RAM equal to console proxy.
- Fixes to several marvin based smoke tests, especially rVR related
tests. rVR tests to consider 3*advert_int+skew timeout before status
is checked.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This ports PR #1470 by @remibergsma.
Make the generated json files unique to prevent concurrency issues:
The json files now have UUIDs to prevent them from getting overwritten
before they've been executed. Prevents config to be pushed to the wrong
router.
2016-02-25 18:32:23,797 DEBUG [c.c.a.t.Request] (AgentManager-Handler-1:null) (logid:) Seq 2-4684025087442026584: Processing: { Ans: , MgmtId: 90520732674657, via: 2, Ver: v1, Flags: 10, [{"com.cloud.agent.api.routing.GroupA
nswer":{"results":["null - success: null","null - success: [INFO] update_config.py :: Processing incoming file => vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b\n[INFO] Processing JSON file vm_dhcp_entry.json.4ea4506
1-2efb-4467-8eaa-db3d77fb0a7b\n"],"result":true,"wait":0}}] }
On the router:
2016-02-25 18:32:23,416 merge.py __moveFile:298 Processed file written to /var/cache/cloud/processed/vm_dhcp_entry.json.4ea45061-2efb-4467-8eaa-db3d77fb0a7b.gz
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Refactor cloud-early-config and make appliance specific scripts
- Make patching work without requiring restart of appliance and remove
postinit script
- Migrate to systemd, speedup booting/loading
- Takes about 5-15s to boot on KVM, and 10-30seconds for VMware and XenServer
- Appliance boots and works on KVM, VMware, XenServer and HyperV
- Update Debian9 ISO url with sha512 checksum
- Speedup console proxy service launch
- Enable additional kernel modules
- Remove unknown ssh key
- Update vhd-util URL as previous URL was down
- Enable sshd by default
- Use hostnamectl to add hostname
- Disable services by default
- Use existing log4j xml, patching not necessary by cloud-early-config
- Several minor fixes and file refactorings, removed dead code/files
- Removes inserv
- Fix dnsmasq config syntax
- Fix haproxy config syntax
- Fix smoke tests and improve performance
- Fix apache pid file path in cloud.monitoring per the new template
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Load the nf_conntrack_ipv6 module for IPv6 connection tracking on SSVM
- Move systemd services to /etc and enable services after they have been
installed
- Disable most services by default and enable in cloud-early-config
- Start services after enabling them using systemd
- In addition remove /etc/init.d/cloud as this is no longer needed and
done by systemd
- Accept DOS/MBR as file format for ISO images as well
Under Debian 7 the 'file' command would return:
debian-9.1.0-amd64-netinst.iso: ISO 9660 CD-ROM filesystem data UDF filesystem data
Under Debian 9 however it will return
debian-9.1.0-amd64-netinst.iso: DOS/MBR boot sector
This would make the HTTPTemplateDownloader in the Secondary Storage VM refuse the ISO as
a valid template because it's not a correct format.
Changes this behavior so that it accepts both.
This allows us to use Debian 9 as a System VM template.
Not sure though if enabling them is enough for systemd to still start them
on first boot
Signed-off-by: Wido den Hollander <wido@widodh.nl>
SystemVM changes to work on Debian 9
- Migrate away from chkconfig to systemctl
- Remove xenstore-utils override deb pkg
- Fix runlevel in sysv scripts for systemd
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
We found bug in ACL rules for PrivateGateway for VPC
At a glance - rules not applied - switching Allow All or Deny All (default ACL) - showed as completed - but rules missed.
Result - traffic via PrivateGateway blocked by next DROP rule in next chains
How to reproduce:
Enable PrivateGateway for Cloudstack
Create VPC
Provision new PrivateGateway inside VPC with some VLAN
Change ACL (optional step to show that problem not in initial configuration but in config itself)
Expected:
ACL rules applied (inserted) into correspondig ACL_INBOUND/OUTBOUND chanins for PrivateGateway interface (ethX) based on ACL which user choose
Current:
No rules inserted. ACL_INBOUND/OUTBOUND_ethX - empty. Traffic blocked by next DROP rule in FORWARD chain
Affect - all our corporate customers blocked with access to their own nets via PG and vice-versa.
Root cause:
Issue happened because of CsNetFilter.py logic for inserting rules for ACL_INBOUND/OUTBOUND chains.
We choose rule numebr to isnert right before last DROP rule - but forget about fact - that if chain empty - we also return 0 as insert position. Which not true for iptables - numeration started from 0.
So we need very small patch to handle this special case - if number of rules inside chain equal to zero - return 1, else - return count of rules inside chain.
It's found only one - just because be default for PrivateGateway - we didn't insert any "service rules" (if SourceNat for PrivateGteway not ticked) - and we have by default empty ACL_INBOUND/OUTBOUND chains. Because same insert happened for all VPC networks (but when we call this insert - we already have at least 1 rule inside chains - and we successfully can process)
