While searching for existing route, don't use the throw keyword when
using the cmd with `ip route show`.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* Increase lease time to infinite
Lease time set to effectively infinite (36000+ days) since we fully control VM lifecycle via CloudStack
Infinite time helps avoid some edge cases which could cause DHCPNAK being sent to VMs since
(RHEL) system lose routes when they receive DHCPNAK
When VM is expunged, it's active lease and DHCP/DNS config is properly removed from related files in VR.
* desc fix
In some virtual routers, 'hostname -f' returns 'localhost'. The hostname is also 'localhost' in `/var/log/messages`. This change can fix the issue in new VRs.
This does not remove VM entries in dbags when hostnames match. The
current codebase already removes entry when a VM is stopped/removed so
we don't need to handle lazy removal. This will allow a VM on
multiple-tiers in a VPC to get dns/dhcp rules as expected.
This also fixes the issue of dhcp_release based on a specific interface and
removes dhcp/dns entry when a nic is removed on a guest VM.
Fixes#3273
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Since the CloudStack virtual router was redesigned on version 4.6 it has been observed that the DHCP leases file is not persistent across network operations. This causes conflicts on guest VMs static IPs, causing these static IPs to not be renewed by the DHCP server running on isolated and VPC networks' virtual routers (dnsmasq). On stopping or destroying a VM, its dhcp/dns records are not removed from the virtual router causing ghost effects.
Fixes#3272Fixes#3354
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This PR resolves 2 issues related to Virtual Routers with multiple public interfaces, and works around a third.
- Fixes#3353 - Adds missing throw routes for eth0/eth1 to eth3+ when there are >1 public IPs
- Fixes#3168 - Incorrect marks set on some static NAT rules (some code references were changed from hex(int(interfacenum)) to hex(100 + int(interfacenum)) - this change just adds the remaining ones
- Fixes#3352 - Work around that sends Gratuitous ARP messages when a HA VR becomes master to work around the problem of the MAC address being different between HA VRs. If that issue is fixed properly (i.e. a database entry for the subsequent interfaces so they can be static) then this is unnecessary, though should not cause any problems.
Previously, the ethernet device index was used as rt_table index and
packet marking id/integer. With eth0 that is sometimes used as link-local
interface, the rt_table index `0` would fail as `0` is already defined
as a catchall (unspecified). The fwmarking on packets on eth0 with 0x0
would also fail. This fixes the routing issues, by adding 100 to the
ethernet device index so the value is a non-zero, for example then the
relationship between rt_table index and ethernet would be like:
100 -> Table_eth0 -> eth0 -> fwmark 100 or 0x64
101 -> Table_eth1 -> eth1 -> fwmark 101 or 0x65
102 -> Table_eth2 -> eth2 -> fwmark 102 or 0x66
This would maintain the legacy design of routing based on packet mark
and appropriate routing table rules per table/ids. This also fixes a
minor NPE issue around listing of snapshots.
This also backports fixes to smoketests from master.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
Using Source NAT option on Private Gateway does not work
This fixes#2680
## Description
<!--- Describe your changes in detail -->
When you use the Source NAT feature of Private Gateways on a VPC. This should Source NAT all traffic from CloudStack VMs going towards IPs reachable through Private Gateways.
This change in this PR, stops adding the Source CIDR to SNAT rules. This should be discussed/reviewed, but i can see no reason why the Source CIDR is needed. There can only be one SNAT IP per interface, except for Static (one-to-one) NATs, which still work with this change in place. The outbound interface is what matters in the rule.
<!-- For new features, provide link to FS, dev ML discussion etc. -->
<!-- In case of bug fix, the expected and actual behaviours, steps to reproduce. -->
##### SUMMARY
<!-- Explain the problem/feature briefly -->
There is a bug in the Private Gateway functionality, when Source NAT is enabled for the Private Gateway. When the SNAT is added to iptables, it has the source CIDR of the private gateway subnet. Since no VMs live in that private gateway subnet, the SNAT doesn’t work.
##### STEPS TO REPRODUCE
<!--
For bugs, show exactly how to reproduce the problem, using a minimal test-case. Use Screenshots if accurate.
For new features, show how the feature would be used.
-->
<!-- Paste example playbooks or commands between quotes below -->
Below is an example:
- VMs have IP addresses in the 10.0.0.0/24 subnet.
- The Private Gateway address is 10.101.141.2/30
In the outputs below, the SOURCE field for the new SNAT (eth3) only matches if the source is 10.101.141.0/30. Since the VM has an IP address in 10.0.0.0/24, the VMs don’t get SNAT’d as they should when talking across the private gateway. The SOURCE should be set to ANYWHERE.
##### BEFORE ADDING PRIVATE GATEWAY
~~~
Chain POSTROUTING (policy ACCEPT 1 packets, 52 bytes)
pkts bytes target prot opt in out source destination
2 736 SNAT all -- any eth2 10.0.0.0/24 anywhere to:10.0.0.1
16 1039 SNAT all -- any eth1 anywhere anywhere to:46.99.52.18
~~~
<!-- You can also paste gist.github.com links for larger files -->
##### EXPECTED RESULTS
<!-- What did you expect to happen when running the steps above? -->
~~~
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 SNAT all -- any eth3 anywhere anywhere to:10.101.141.2
2 736 SNAT all -- any eth2 anywhere anywhere to:10.0.0.1
23 1515 SNAT all -- any eth1 anywhere anywhere to:46.99.52.18
~~~
##### ACTUAL RESULTS
<!-- What actually happened? -->
<!-- Paste verbatim command output between quotes below -->
~~~
Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 SNAT all -- any eth3 10.101.141.0/30 anywhere to:10.101.141.2
2 736 SNAT all -- any eth2 10.0.0.0/24 anywhere to:10.0.0.1
23 1515 SNAT all -- any eth1 anywhere anywhere to:46.99.52.18
~~~
## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] New feature (non-breaking change which adds functionality)
- [X] Bug fix (non-breaking change which fixes an issue)
- [ ] Enhancement (improves an existing feature and functionality)
- [ ] Cleanup (Code refactoring and cleanup, that may add test cases)
## GitHub Issue/PRs
<!-- If this PR is to fix an issue or another PR on GH, uncomment the section and provide the id of issue/PR -->
<!-- When "Fixes: #<id>" is specified, the issue/PR will automatically be closed when this PR gets merged -->
<!-- For addressing multiple issues/PRs, use multiple "Fixes: #<id>" -->
Fixes: #2680
## Screenshots (if appropriate):
## How Has This Been Tested?
<!-- Please describe in detail how you tested your changes. -->
<!-- Include details of your testing environment, and the tests you ran to -->
<!-- see how your change affects other areas of the code, etc. -->
## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [x] I have read the [CONTRIBUTING](https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md) document.
- [x] My code follows the code style of this project.
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
Testing
- [ ] I have added tests to cover my changes.
- [ ] All relevant new and existing integration tests have passed.
- [ ] A full integration testsuite with all test that can run on my environment has passed.
This ensures that password server runs on the dhcpserver identifier
IP which is the not the VRRP virtual (10.1.1.1) IP by default but
the actual ip of the interface. When dhcp client discovery is made,
the `dhcp-server-identifier` contains the non VIP address that is
used by password reset script to query guest VM password.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This introduces a rolling restart of VRs when networks are restarted
with cleanup option for isolated and VPC networks. A make redundant option is
shown for isolated networks now in UI.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This adds support for XenServer 7.3 and 7.4, and XCP-ng 7.4 version as hypervisor hosts. Fixes#2523.
This also fixes the issue of 4.11 VRs stuck in starting for up-to 10mins, before they come up online.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This fixes routing table rule setup regression to correctly router
marked packets based on interface related ip route tables. This thereby
fixes the access of VMs in the same VPC using NAT/SNAT public IPs.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* dependencies update
* Add extra blank line required by ...!?
* fix W605 invalid escape sequence and more blank lines
* print all installed python packages versions
* CLOUDSTACK-10317: Fix SNAT rules for additional public nics
This allows networks with additional public nics to have correct
SNAT iptables rules applied on configuration.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
* update based on Wei's suggested change
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This might (and does block) in certain situations on the VR as
also explained in the Python documentation:
https://docs.python.org/2/library/subprocess.html#subprocess.Popen.wait
Warning This will deadlock when using stdout=PIPE and/or stderr=PIPE
and the child process generates enough output to a pipe such that
it blocks waiting for the OS pipe buffer to accept more data.
Use communicate() to avoid that.
Using the check_output function handles most of this for us and
also provides better error handling.
Signed-off-by: Wido den Hollander <wido@widodh.nl>
Delete dnsmasq's leases file when dnsmasq is restarted to avoid it
use old ip-mac-address-vm mapping leases.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
In case of isolated, both self.config.is_vpc() and self.config.is_router() are false,
but self.config.is_dhcp() is true.
Moved the password server logic to the `if has_metadata` block,
as this is valid for all 3 systemvm types.
This includes test related fixes and code review fixes based on
reviews from @rafaelweingartner, @marcaurele, @wido and @DaanHoogland.
This also includes VMware disk-resize limitation bug fix based on comments
from @sateesh-chodapuneedi and @priyankparihar.
This also includes the final changes to systemvmtemplate and fixes to
code based on issues found via test failures.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
This fixes test failures around VMware with the new systemvmtemplate.
In addition:
- Does not skip rVR related test cases for VMware
- Removes rc.local
- Processes unprocessed cmd_line.json
- Fixed NPEs around VMware tests/code
- On VMware, use udevadm to reconfigure nic/mac address than rebooting
- Fix proper acpi shutdown script for faster systemvm shutdowns
- Give at least 256MB of swap for VRs to avoid OOM on VMware
- Fixes smoke tests for environment related failures
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Several systemvmtemplate optimizations
- Uses new macchinina template for running smoke tests
- Switch to latest Debian 9.3.0 release for systemvmtemplate
- Introduce a new `get_test_template` that uses tiny test template
such as macchinina as defined test_data.py
- rVR related fixes and improvements
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
- Refactors and simplifies systemvm codebase file structures keeping
the same resultant systemvm.iso packaging
- Password server systemd script and new postinit script that runs
before sshd starts
- Fixes to keepalived and conntrackd config to make rVRs work again
- New /etc/issue featuring ascii based cloudmonkey logo/message and
systemvmtemplate version
- SystemVM python codebase linted and tested. Added pylint/pep to
Travis.
- iptables re-application fixes for non-VR systemvms.
- SystemVM template build fixes.
- Default secondary storage vm service offering boosted to have 2vCPUs
and RAM equal to console proxy.
- Fixes to several marvin based smoke tests, especially rVR related
tests. rVR tests to consider 3*advert_int+skew timeout before status
is checked.
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
