mirror of
				https://github.com/vyos/vyos-documentation.git
				synced 2025-11-04 00:02:05 +01:00 
			
		
		
		
	
		
			
				
	
	
		
			255 lines
		
	
	
		
			9.0 KiB
		
	
	
	
		
			ReStructuredText
		
	
	
	
	
	
			
		
		
	
	
			255 lines
		
	
	
		
			9.0 KiB
		
	
	
	
		
			ReStructuredText
		
	
	
	
	
	
 | 
						|
####################
 | 
						|
L3VPN EVPN with VyOS
 | 
						|
####################
 | 
						|
 | 
						|
| Testdate: 2021-11-25
 | 
						|
| Version: 1.4-rolling-202111240711
 | 
						|
 | 
						|
I spun up a new lab in EVE-NG, which represents this as the
 | 
						|
"Foo Bar - Service Provider Inc." that has 3 points of presence (PoP) in random
 | 
						|
datacenters/sites named PE1, PE2, and PE3. Each PoP aggregates at least two
 | 
						|
customers.
 | 
						|
 | 
						|
I named the customers blue, red and green which is common practice in
 | 
						|
VRF (Virtual Routing and Forwarding) documentation scenarios.
 | 
						|
 | 
						|
* PE1 is located in an industrial area that holds multiple office buildings.
 | 
						|
  All customers have a site in this area.
 | 
						|
* PE2 is located in a smaller area where by coincidence two customers
 | 
						|
  (blue and red) share an office building.
 | 
						|
* PE3 is located in a smaller area where by coincidence two customers
 | 
						|
  (blue and green) are located.
 | 
						|
 | 
						|
**************
 | 
						|
Management VRF
 | 
						|
**************
 | 
						|
 | 
						|
A brief excursion into VRFs: This has been one of the longest-standing feature
 | 
						|
requests of VyOS (dating back to 2016) which can be described as
 | 
						|
"a VLAN for layer 2 is what a VRF is for layer 3". 
 | 
						|
With VRFs, a router/system can hold multiple, isolated routing tables on the
 | 
						|
same system. If you wonder what's the difference between multiple tables that
 | 
						|
people used for policy-based routing since forever, it's that a VRF also
 | 
						|
isolates connected routes rather than just static and dynamically learned
 | 
						|
routes, so it allows NICs in different VRFs to use conflicting network
 | 
						|
ranges without issues.
 | 
						|
 | 
						|
VyOS 1.3 added initial support for VRFs (including IPv4/IPv6 static routing)
 | 
						|
and VyOS 1.4 now enables full dynamic routing protocol support for
 | 
						|
OSPF, IS-IS, and BGP for individual VRFs.
 | 
						|
 | 
						|
The lab I built is using a VRF (called **mgmt**) to provide out-of-band
 | 
						|
SSH access to the PE (Provider Edge) routers. 
 | 
						|
 | 
						|
.. literalinclude:: _include/PE1.conf
 | 
						|
   :language: none
 | 
						|
   :lines: 1-6
 | 
						|
 | 
						|
 | 
						|
********
 | 
						|
Topology
 | 
						|
********
 | 
						|
 | 
						|
We use the following network topology in this example:
 | 
						|
 | 
						|
.. image:: _include/topology.png
 | 
						|
  :alt: L3VPN EVPN with VyOS topology image
 | 
						|
 | 
						|
 | 
						|
************
 | 
						|
Core network
 | 
						|
************
 | 
						|
 | 
						|
I chose to run OSPF as the IGP (Interior Gateway Protocol).
 | 
						|
All required BGP sessions are established via a dummy interfaces
 | 
						|
(similar to the loopback, but in Linux you can have only one loopback,
 | 
						|
while there can be many dummy interfaces) on the PE routers. In case of a link
 | 
						|
failure, traffic is diverted in the other direction in this triangle setup and
 | 
						|
BGP sessions will not go down. One could even enable
 | 
						|
BFD (Bidirectional Forwarding Detection) on the links for a faster
 | 
						|
failover and resilience in the network.
 | 
						|
 | 
						|
Regular VyOS users will notice that the BGP syntax has changed in VyOS 1.4 from
 | 
						|
even the prior post about this subject. This is due to T1711, where it was
 | 
						|
finally decided to get rid of the redundant BGP ASN (Autonomous System Number)
 | 
						|
specification on the CLI and move it to a single leaf node
 | 
						|
(set protocols bgp local-as).
 | 
						|
 | 
						|
It's important to note that all your existing configurations will be migrated
 | 
						|
automatically on image upgrade. Nothing to do on your side.
 | 
						|
 | 
						|
PE1
 | 
						|
 | 
						|
.. literalinclude:: _include/PE1.conf
 | 
						|
   :language: none
 | 
						|
   :lines: 8-38
 | 
						|
 | 
						|
PE2
 | 
						|
 | 
						|
.. literalinclude:: _include/PE2.conf
 | 
						|
   :language: none
 | 
						|
   :lines: 8-38
 | 
						|
 | 
						|
PE3
 | 
						|
 | 
						|
.. literalinclude:: _include/PE3.conf
 | 
						|
   :language: none
 | 
						|
   :lines: 8-38
 | 
						|
 | 
						|
 | 
						|
**********************
 | 
						|
Tenant networks (VRFs)
 | 
						|
**********************
 | 
						|
 | 
						|
Once all routers can be safely remotely managed and the core network is
 | 
						|
operational, we can now setup the tenant networks.
 | 
						|
 | 
						|
Every tenant is assigned an individual VRF that would support overlapping
 | 
						|
address ranges for customers blue, red and green. In our example,
 | 
						|
we do not use overlapping ranges to make it easier when showing debug commands.
 | 
						|
 | 
						|
Thus you can easily match it to one of the devices/networks below.
 | 
						|
 | 
						|
Every router that provides access to a customer network needs to have the
 | 
						|
customer network (VRF + VNI) configured. To make our own lives easier,
 | 
						|
we utilize the same VRF table id (local routing table number) and
 | 
						|
VNI (Virtual Network Identifier) per tenant on all our routers.
 | 
						|
 | 
						|
* blue uses local routing table id and VNI 2000
 | 
						|
* red uses local routing table id and VNI 3000
 | 
						|
* green uses local routing table id and VNI 4000
 | 
						|
 | 
						|
PE1
 | 
						|
 | 
						|
.. literalinclude:: _include/PE1.conf
 | 
						|
   :language: none
 | 
						|
   :lines: 40-96
 | 
						|
 | 
						|
PE2
 | 
						|
 | 
						|
.. literalinclude:: _include/PE2.conf
 | 
						|
   :language: none
 | 
						|
   :lines: 40-89
 | 
						|
 | 
						|
PE3
 | 
						|
 | 
						|
.. literalinclude:: _include/PE3.conf
 | 
						|
   :language: none
 | 
						|
   :lines: 40-89
 | 
						|
 | 
						|
*********************
 | 
						|
Testing and debugging
 | 
						|
*********************
 | 
						|
 | 
						|
You managed to come this far, now we want to see the network and routing
 | 
						|
tables in action.
 | 
						|
 | 
						|
Show routes for all VRFs
 | 
						|
 | 
						|
 | 
						|
.. code-block:: none
 | 
						|
 | 
						|
   vyos@PE1:~$ show ip route vrf all
 | 
						|
   Codes: K - kernel route, C - connected, S - static, R - RIP,
 | 
						|
          O - OSPF, I - IS-IS, B - BGP, E - EIGRP, N - NHRP,
 | 
						|
          T - Table, v - VNC, V - VNC-Direct, A - Babel, D - SHARP,
 | 
						|
          F - PBR, f - OpenFabric,
 | 
						|
          > - selected route, * - FIB route, q - queued, r - rejected, b - backup
 | 
						|
   
 | 
						|
   VRF blue:
 | 
						|
   K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:00:59
 | 
						|
   C>* 10.1.1.0/24 is directly connected, br2000, 00:00:58
 | 
						|
   B>* 10.1.2.0/24 [200/0] via 172.29.255.2, br2000 onlink, weight 1, 00:00:34
 | 
						|
   B>* 10.1.3.0/24 [200/0] via 172.29.255.3, br2000 onlink, weight 1, 00:00:34
 | 
						|
   
 | 
						|
   VRF default:
 | 
						|
   O   172.29.0.2/31 [110/1] is directly connected, eth1, weight 1, 00:00:55
 | 
						|
   C>* 172.29.0.2/31 is directly connected, eth1, 00:00:58
 | 
						|
   O>* 172.29.0.4/31 [110/2] via 172.29.0.3, eth1, weight 1, 00:00:31
 | 
						|
     *                       via 172.29.0.7, eth3, weight 1, 00:00:31
 | 
						|
   O   172.29.0.6/31 [110/1] is directly connected, eth3, weight 1, 00:00:55
 | 
						|
   C>* 172.29.0.6/31 is directly connected, eth3, 00:00:58
 | 
						|
   C>* 172.29.255.1/32 is directly connected, dum0, 00:00:59
 | 
						|
   O>* 172.29.255.2/32 [110/20] via 172.29.0.3, eth1, weight 1, 00:00:35
 | 
						|
   O>* 172.29.255.3/32 [110/20] via 172.29.0.7, eth3, weight 1, 00:00:30
 | 
						|
   
 | 
						|
   VRF green:
 | 
						|
   K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:00:59
 | 
						|
   C>* 10.3.1.0/24 is directly connected, br4000, 00:00:58
 | 
						|
   B>* 10.3.3.0/24 [200/0] via 172.29.255.3, br4000 onlink, weight 1, 00:00:34
 | 
						|
   
 | 
						|
   VRF mgmt:
 | 
						|
   S>* 0.0.0.0/0 [210/0] via 10.100.0.1, eth0, weight 1, 00:01:56
 | 
						|
   K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:01:59
 | 
						|
   C>* 10.100.0.0/24 is directly connected, eth0, 00:01:57
 | 
						|
   
 | 
						|
   VRF red:
 | 
						|
   K>* 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:00:59
 | 
						|
   C>* 10.2.1.0/24 is directly connected, br3000, 00:00:58
 | 
						|
   B>* 10.2.2.0/24 [200/0] via 172.29.255.2, br3000 onlink, weight 1, 00:00:34
 | 
						|
 | 
						|
Information about Ethernet Virtual Private Networks
 | 
						|
 | 
						|
 | 
						|
.. code-block:: none
 | 
						|
 | 
						|
   vyos@PE1:~$ show bgp l2vpn evpn
 | 
						|
   BGP table version is 1, local router ID is 172.29.255.1
 | 
						|
   Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
 | 
						|
   Origin codes: i - IGP, e - EGP, ? - incomplete
 | 
						|
   EVPN type-1 prefix: [1]:[ESI]:[EthTag]:[IPlen]:[VTEP-IP]
 | 
						|
   EVPN type-2 prefix: [2]:[EthTag]:[MAClen]:[MAC]:[IPlen]:[IP]
 | 
						|
   EVPN type-3 prefix: [3]:[EthTag]:[IPlen]:[OrigIP]
 | 
						|
   EVPN type-4 prefix: [4]:[ESI]:[IPlen]:[OrigIP]
 | 
						|
   EVPN type-5 prefix: [5]:[EthTag]:[IPlen]:[IP]
 | 
						|
   
 | 
						|
      Network          Next Hop            Metric LocPrf Weight Path
 | 
						|
   Route Distinguisher: 10.1.1.1:5
 | 
						|
   *> [5]:[0]:[24]:[10.1.1.0]
 | 
						|
                       172.29.255.1             0         32768 ?
 | 
						|
                       ET:8 RT:100:2000 Rmac:50:00:00:01:00:04
 | 
						|
   Route Distinguisher: 10.1.2.1:4
 | 
						|
   *>i[5]:[0]:[24]:[10.1.2.0]
 | 
						|
                       172.29.255.2             0    100      0 ?
 | 
						|
                       RT:100:2000 ET:8 Rmac:4a:da:66:c7:5a:54
 | 
						|
   Route Distinguisher: 10.1.3.1:4
 | 
						|
   *>i[5]:[0]:[24]:[10.1.3.0]
 | 
						|
                       172.29.255.3             0    100      0 ?
 | 
						|
                       RT:100:2000 ET:8 Rmac:50:00:00:03:00:04
 | 
						|
   Route Distinguisher: 10.2.1.1:6
 | 
						|
   *> [5]:[0]:[24]:[10.2.1.0]
 | 
						|
                       172.29.255.1             0         32768 ?
 | 
						|
                       ET:8 RT:100:3000 Rmac:50:00:00:01:00:05
 | 
						|
   Route Distinguisher: 10.2.2.1:5
 | 
						|
   *>i[5]:[0]:[24]:[10.2.2.0]
 | 
						|
                       172.29.255.2             0    100      0 ?
 | 
						|
                       RT:100:3000 ET:8 Rmac:1a:c4:c5:ec:b3:e6
 | 
						|
   Route Distinguisher: 10.3.1.1:7
 | 
						|
   *> [5]:[0]:[24]:[10.3.1.0]
 | 
						|
                       172.29.255.1             0         32768 ?
 | 
						|
                       ET:8 RT:100:4000 Rmac:50:00:00:01:00:06
 | 
						|
   Route Distinguisher: 10.3.3.1:6
 | 
						|
   *>i[5]:[0]:[24]:[10.3.3.0]
 | 
						|
                       172.29.255.3             0    100      0 ?
 | 
						|
                       RT:100:4000 ET:8 Rmac:0a:61:a1:5c:7b:14
 | 
						|
   
 | 
						|
   Displayed 7 out of 7 total prefixes
 | 
						|
 | 
						|
If we need to retrieve information about a specific host/network inside
 | 
						|
the EVPN network we need to run
 | 
						|
 | 
						|
 | 
						|
.. code-block:: none
 | 
						|
 | 
						|
   vyos@PE2:~$ show bgp l2vpn evpn 10.3.1.10
 | 
						|
   BGP routing table entry for 10.3.1.1:7:[5]:[0]:[24]:[10.3.1.0]
 | 
						|
   Paths: (1 available, best #1)
 | 
						|
     Not advertised to any peer
 | 
						|
     Route [5]:[0]:[24]:[10.3.1.0] VNI 4000
 | 
						|
     Local
 | 
						|
       172.29.255.1 (metric 20) from 172.29.255.1 (172.29.255.1)
 | 
						|
         Origin incomplete, metric 0, localpref 100, valid, internal, best (First path received)
 | 
						|
         Extended Community: RT:100:4000 ET:8 Rmac:50:00:00:01:00:06
 | 
						|
         Last update: Thu Nov 25 19:45:06 2021
 |