mirror of
https://github.com/vyos/vyos-documentation.git
synced 2025-10-26 08:41:46 +01:00
b6e494eca6
Added warning about changing VRRP configuration in scripts. Added information about health-check scripts.
489 lines
16 KiB
ReStructuredText
489 lines
16 KiB
ReStructuredText
:lastproofread: 2021-06-30
|
|
|
|
.. _high-availability:
|
|
|
|
High availability
|
|
=================
|
|
|
|
VRRP (Virtual Router Redundancy Protocol) provides active/backup redundancy for
|
|
routers. Every VRRP router has a physical IP/IPv6 address, and a virtual
|
|
address. On startup, routers elect the master, and the router with the highest
|
|
priority becomes the master and assigns the virtual address to its interface.
|
|
All routers with lower priorities become backup routers. The master then starts
|
|
sending keepalive packets to notify other routers that it's available. If the
|
|
master fails and stops sending keepalive packets, the router with the next
|
|
highest priority becomes the new master and takes over the virtual address.
|
|
|
|
VRRP keepalive packets use multicast, and VRRP setups are limited to a single
|
|
datalink layer segment. You can setup multiple VRRP groups
|
|
(also called virtual routers). Virtual routers are identified by a
|
|
VRID (Virtual Router IDentifier). If you setup multiple groups on the same
|
|
interface, their VRIDs must be unique if they use the same address family,
|
|
but it's possible (even if not recommended for readability reasons) to use
|
|
duplicate VRIDs on different interfaces.
|
|
|
|
Basic setup
|
|
-----------
|
|
|
|
VRRP groups are created with the
|
|
``set high-availability vrrp group $GROUP_NAME`` commands. The required
|
|
parameters are interface, vrid, and address.
|
|
|
|
minimal config
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo vrid 10
|
|
set high-availability vrrp group Foo interface eth0
|
|
set high-availability vrrp group Foo address 192.0.2.1/24
|
|
|
|
You can verify your VRRP group status with the operational mode
|
|
``run show vrrp`` command:
|
|
|
|
.. code-block:: none
|
|
|
|
vyos@vyos# run show vrrp
|
|
Name Interface VRID State Last Transition
|
|
---------- ----------- ------ ------- -----------------
|
|
Foo eth1 10 MASTER 2s
|
|
|
|
IPv6 support
|
|
------------
|
|
|
|
The ``address`` parameter can be either an IPv4 or IPv6 address, but you can
|
|
not mix IPv4 and IPv6 in the same group, and will need to create groups with
|
|
different VRIDs specially for IPv4 and IPv6.
|
|
If you want to use IPv4 + IPv6 address you can use option ``excluded-address``
|
|
|
|
Address
|
|
-------
|
|
The ``address`` can be configured either on the VRRP interface or on not VRRP
|
|
interface.
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo address 192.0.2.1/24
|
|
set high-availability vrrp group Foo address 203.0.113.22/24 interface eth2
|
|
set high-availability vrrp group Foo address 198.51.100.33/24 interface eth3
|
|
|
|
Disabling a VRRP group
|
|
----------------------
|
|
|
|
You can disable a VRRP group with ``disable`` option:
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo disable
|
|
|
|
A disabled group will be removed from the VRRP process and your router will not
|
|
participate in VRRP for that VRID. It will disappear from operational mode
|
|
commands output, rather than enter the backup state.
|
|
|
|
Exclude address
|
|
---------------
|
|
|
|
Exclude IP addresses from ``VRRP packets``. This option ``excluded-address`` is
|
|
used when you want to set IPv4 + IPv6 addresses on the same virtual interface
|
|
or when used more than 20 IP addresses.
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo excluded-address '203.0.113.254/24'
|
|
set high-availability vrrp group Foo excluded-address '2001:db8:aa::1/64'
|
|
set high-availability vrrp group Foo excluded-address '2001:db8:22::1/64'
|
|
|
|
Setting VRRP group priority
|
|
---------------------------
|
|
|
|
VRRP priority can be set with ``priority`` option:
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo priority 200
|
|
|
|
The priority must be an integer number from 1 to 255. Higher priority value
|
|
increases router's precedence in the master elections.
|
|
|
|
Sync groups
|
|
-----------
|
|
|
|
A sync group allows VRRP groups to transition together.
|
|
|
|
.. code-block:: none
|
|
|
|
edit high-availability vrrp
|
|
set sync-group MAIN member VLAN9
|
|
set sync-group MAIN member VLAN20
|
|
|
|
In the following example, when VLAN9 transitions, VLAN20 will also transition:
|
|
|
|
.. code-block:: none
|
|
|
|
vrrp {
|
|
group VLAN9 {
|
|
interface eth0.9
|
|
address 10.9.1.1/24
|
|
priority 200
|
|
vrid 9
|
|
}
|
|
group VLAN20 {
|
|
interface eth0.20
|
|
priority 200
|
|
address 10.20.20.1/24
|
|
vrid 20
|
|
}
|
|
sync-group MAIN {
|
|
member VLAN20
|
|
member VLAN9
|
|
}
|
|
}
|
|
|
|
|
|
.. warning:: All items in a sync group should be similarly configured.
|
|
If one VRRP group is set to a different preemption delay or priority,
|
|
it would result in an endless transition loop.
|
|
|
|
|
|
Preemption
|
|
----------
|
|
|
|
VRRP can use two modes: preemptive and non-preemptive. In the preemptive mode,
|
|
if a router with a higher priority fails and then comes back, routers with lower
|
|
priority will give up their master status. In non-preemptive mode, the newly
|
|
elected master will keep the master status and the virtual address indefinitely.
|
|
|
|
By default VRRP uses preemption. You can disable it with the "no-preempt"
|
|
option:
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo no-preempt
|
|
|
|
You can also configure the time interval for preemption with the "preempt-delay"
|
|
option. For example, to set the higher priority router to take over in 180
|
|
seconds, use:
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo preempt-delay 180
|
|
|
|
Track
|
|
-----
|
|
|
|
Track option to track non VRRP interface states. VRRP changes status to
|
|
``FAULT`` if one of the track interfaces in state ``down``.
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo track interface eth0
|
|
set high-availability vrrp group Foo track interface eth1
|
|
|
|
Ignore VRRP main interface faults
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo track exclude-vrrp-interface
|
|
|
|
Unicast VRRP
|
|
------------
|
|
|
|
By default VRRP uses multicast packets. If your network does not support
|
|
multicast for whatever reason, you can make VRRP use unicast communication
|
|
instead.
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo peer-address 192.0.2.10
|
|
set high-availability vrrp group Foo hello-source-address 192.0.2.15
|
|
|
|
rfc3768-compatibility
|
|
---------------------
|
|
|
|
RFC 3768 defines a virtual MAC address to each VRRP virtual router.
|
|
This virtual router MAC address will be used as the source in all periodic VRRP
|
|
messages sent by the active node. When the rfc3768-compatibility option is set,
|
|
a new VRRP interface is created, to which the MAC address and the virtual IP
|
|
address is automatically assigned.
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo rfc3768-compatibility
|
|
|
|
Verification
|
|
|
|
.. code-block:: none
|
|
|
|
$show interfaces ethernet eth0v10
|
|
eth0v10@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
|
|
state UP group default qlen 1000
|
|
link/ether 00:00:5e:00:01:0a brd ff:ff:ff:ff:ff:ff
|
|
inet 172.25.0.247/16 scope global eth0v10
|
|
valid_lft forever preferred_lft forever
|
|
|
|
.. warning:: RFC 3768 creates a virtual interface. If you want to apply
|
|
the destination NAT rule to the traffic sent to the virtual MAC, set
|
|
the created virtual interface as `inbound-interface`.
|
|
|
|
Global options
|
|
--------------
|
|
|
|
On most scenarios, there's no need to change specific parameters, and using
|
|
default configuration is enough. But there are cases were extra configuration
|
|
is needed.
|
|
|
|
.. cfgcmd:: set high-availability vrrp global-parameters startup_delay <1-600>
|
|
|
|
This option specifies a delay in seconds before vrrp instances start up
|
|
after keepalived starts.
|
|
|
|
Gratuitous ARP
|
|
--------------
|
|
|
|
These configuration is not mandatory and in most cases there's no
|
|
need to configure it. But if necessary, Gratuitous ARP can be configured in
|
|
``global-parameters`` and/or in ``group`` section.
|
|
|
|
.. cfgcmd:: set high-availability vrrp global-parameters garp interval
|
|
<0.000-1000>
|
|
|
|
.. cfgcmd:: set high-availability vrrp group <name> garp interval <0.000-1000>
|
|
|
|
Set delay between gratuitous ARP messages sent on an interface.
|
|
|
|
0 if not defined.
|
|
|
|
.. cfgcmd:: set high-availability vrrp global-parameters garp master-delay <1-255>
|
|
|
|
.. cfgcmd:: set high-availability vrrp group <name> garp master-delay <1-255>
|
|
|
|
Set delay for second set of gratuitous ARPs after transition to MASTER.
|
|
|
|
5 if not defined.
|
|
|
|
.. cfgcmd:: set high-availability vrrp global-parameters garp master-refresh
|
|
<1-600>
|
|
|
|
.. cfgcmd:: set high-availability vrrp group <name> garp master-refresh
|
|
<1-600>
|
|
|
|
Set minimum time interval for refreshing gratuitous ARPs while MASTER.
|
|
|
|
0 if not defined, which means no refreshing.
|
|
|
|
.. cfgcmd:: set high-availability vrrp global-parameters garp
|
|
master-refresh-repeat <1-600>
|
|
|
|
.. cfgcmd:: set high-availability vrrp group <name> garp
|
|
master-refresh-repeat <1-600>
|
|
|
|
Set number of gratuitous ARP messages to send at a time while MASTER.
|
|
|
|
1 if not defined.
|
|
|
|
.. cfgcmd:: set high-availability vrrp global-parameters garp master-repeat
|
|
<1-600>
|
|
|
|
.. cfgcmd:: set high-availability vrrp group <name> garp master-repeat
|
|
<1-600>
|
|
|
|
Set number of gratuitous ARP messages to send at a time after transition to
|
|
MASTER.
|
|
|
|
5 if not defined.
|
|
|
|
Version
|
|
-------
|
|
|
|
.. cfgcmd:: set high-availability vrrp global-parameters version 2|3
|
|
|
|
Set the default VRRP version to use. This defaults to 2, but IPv6 instances
|
|
will always use version 3.
|
|
|
|
Scripting
|
|
---------
|
|
|
|
VRRP functionality can be extended with scripts. VyOS supports two kinds of
|
|
scripts: health check scripts and transition scripts. Health check scripts
|
|
execute custom checks in addition to the master router reachability. Transition
|
|
scripts are executed when VRRP state changes from master to backup or fault and
|
|
vice versa and can be used to enable or disable certain services, for example.
|
|
|
|
.. warning:: It is not recommended to change VRRP configuration inside health-check
|
|
and transition scripts.
|
|
|
|
Health check scripts
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
|
|
There is the ability to run an arbitrary script at regular intervals according to health-check
|
|
parameters. If a script returns 0, it indicates success. If a script returns anything
|
|
else, it will indicate that the VRRP instance should enter the FAULT state.
|
|
|
|
This setup will make the VRRP process execute the
|
|
``/config/scripts/vrrp-check.sh script`` every 60 seconds, and transition the
|
|
group to the fault state if it fails (i.e. exits with non-zero status) three
|
|
times:
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo health-check script /config/scripts/vrrp-check.sh
|
|
set high-availability vrrp group Foo health-check interval 60
|
|
set high-availability vrrp group Foo health-check failure-count 3
|
|
|
|
When the vrrp group is a member of the sync group will use only
|
|
the sync group health check script.
|
|
This example shows how to configure it for the sync group:
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp sync-group Bar health-check script /config/scripts/vrrp-check.sh
|
|
set high-availability vrrp sync-group Bar health-check interval 60
|
|
set high-availability vrrp sync-group Bar health-check failure-count 3
|
|
|
|
Transition scripts
|
|
^^^^^^^^^^^^^^^^^^
|
|
|
|
Transition scripts can help you implement various fixups, such as starting and
|
|
stopping services, or even modifying the VyOS config on VRRP transition.
|
|
This setup will make the VRRP process execute the
|
|
``/config/scripts/vrrp-fail.sh`` with argument ``Foo`` when VRRP fails,
|
|
and the ``/config/scripts/vrrp-master.sh`` when the router becomes the master:
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability vrrp group Foo transition-script backup "/config/scripts/vrrp-fail.sh Foo"
|
|
set high-availability vrrp group Foo transition-script fault "/config/scripts/vrrp-fail.sh Foo"
|
|
set high-availability vrrp group Foo transition-script master "/config/scripts/vrrp-master.sh Foo"
|
|
|
|
To know more about scripting, check the :ref:`command-scripting` section.
|
|
|
|
Virtual-server
|
|
--------------
|
|
.. include:: /_include/need_improvement.txt
|
|
|
|
Virtual Server allows to Load-balance traffic destination virtual-address:port
|
|
between several real servers.
|
|
|
|
Algorithm
|
|
^^^^^^^^^
|
|
Load-balancing schedule algorithm:
|
|
|
|
* round-robin
|
|
* weighted-round-robin
|
|
* least-connection
|
|
* weighted-least-connection
|
|
* source-hashing
|
|
* destination-hashing
|
|
* locality-based-least-connection
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability virtual-server 203.0.113.1 algorithm 'least-connection'
|
|
|
|
Forward method
|
|
^^^^^^^^^^^^^^
|
|
* NAT
|
|
* direct
|
|
* tunnel
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability virtual-server 203.0.113.1 forward-method 'nat'
|
|
|
|
Health-check
|
|
^^^^^^^^^^^^
|
|
Custom health-check script allows checking real-server availability
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability virtual-server 203.0.113.1 real-server 192.0.2.11 health-check script <path-to-script>
|
|
|
|
Fwmark
|
|
^^^^^^
|
|
Firewall mark. It possible to loadbalancing traffic based on ``fwmark`` value
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability virtual-server 203.0.113.1 fwmark '111'
|
|
|
|
Real server
|
|
^^^^^^^^^^^
|
|
Real server IP address and port
|
|
|
|
.. code-block:: none
|
|
|
|
set high-availability virtual-server 203.0.113.1 real-server 192.0.2.11 port '80'
|
|
|
|
|
|
Example
|
|
^^^^^^^
|
|
Virtual-server can be configured with VRRP virtual address or without VRRP.
|
|
|
|
In the next example all traffic destined to ``203.0.113.1`` and port ``8280``
|
|
protocol TCP is balanced between 2 real servers ``192.0.2.11`` and
|
|
``192.0.2.12`` to port ``80``
|
|
|
|
Real server is auto-excluded if port check with this server fail.
|
|
|
|
.. code-block:: none
|
|
|
|
set interfaces ethernet eth0 address '203.0.113.11/24'
|
|
set interfaces ethernet eth1 address '192.0.2.1/24'
|
|
set high-availability vrrp group FOO interface 'eth0'
|
|
set high-availability vrrp group FOO no-preempt
|
|
set high-availability vrrp group FOO priority '150'
|
|
set high-availability vrrp group FOO address '203.0.113.1/24'
|
|
set high-availability vrrp group FOO vrid '10'
|
|
|
|
set high-availability virtual-server 203.0.113.1 algorithm 'source-hashing'
|
|
set high-availability virtual-server 203.0.113.1 delay-loop '10'
|
|
set high-availability virtual-server 203.0.113.1 forward-method 'nat'
|
|
set high-availability virtual-server 203.0.113.1 persistence-timeout '180'
|
|
set high-availability virtual-server 203.0.113.1 port '8280'
|
|
set high-availability virtual-server 203.0.113.1 protocol 'tcp'
|
|
set high-availability virtual-server 203.0.113.1 real-server 192.0.2.11 port '80'
|
|
set high-availability virtual-server 203.0.113.1 real-server 192.0.2.12 port '80'
|
|
|
|
|
|
A firewall mark ``fwmark`` allows using multiple ports for high-availability
|
|
virtual-server.
|
|
It uses fwmark value.
|
|
|
|
In this example all traffic destined to ports "80, 2222, 8888" protocol TCP
|
|
marks to fwmark "111" and balanced between 2 real servers.
|
|
Port "0" is required if multiple ports are used.
|
|
|
|
.. code-block:: none
|
|
|
|
set interfaces ethernet eth0 address 'dhcp'
|
|
set interfaces ethernet eth0 description 'WAN'
|
|
set interfaces ethernet eth1 address '192.0.2.1/24'
|
|
set interfaces ethernet eth1 description 'LAN'
|
|
|
|
set policy route PR interface 'eth0'
|
|
set policy route PR rule 10 destination port '80,2222,8888'
|
|
set policy route PR rule 10 protocol 'tcp'
|
|
set policy route PR rule 10 set mark '111'
|
|
|
|
set high-availability virtual-server vyos fwmark '111'
|
|
set high-availability virtual-server vyos protocol 'tcp'
|
|
set high-availability virtual-server vyos real-server 192.0.2.11 health-check script '/config/scripts/check-real-server-first.sh'
|
|
set high-availability virtual-server vyos real-server 192.0.2.11 port '0'
|
|
set high-availability virtual-server vyos real-server 192.0.2.12 health-check script '/config/scripts/check-real-server-second.sh'
|
|
set high-availability virtual-server vyos real-server 192.0.2.12 port '0'
|
|
|
|
set nat source rule 100 outbound-interface name 'eth0'
|
|
set nat source rule 100 source address '192.0.2.0/24'
|
|
set nat source rule 100 translation address 'masquerade'
|
|
|
|
Op-mode check virtual-server status
|
|
|
|
.. code-block:: none
|
|
|
|
vyos@r14:~$ run show virtual-server
|
|
IP Virtual Server version 1.2.1 (size=4096)
|
|
Prot LocalAddress:Port Scheduler Flags
|
|
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
|
|
FWM 111 lc persistent 300
|
|
-> 192.0.2.11:0 Masq 1 0 0
|
|
-> 192.0.2.12:0 Masq 1 1 0
|