vyos/vyos-documentation
1
0
Fork 0
mirror of https://github.com/vyos/vyos-documentation.git synced 2025-10-26 08:41:46 +01:00
Code Issues Packages Projects Releases Wiki Activity
vyos-documentation/docs/troubleshooting/index.rst
rebortg 74b8c6d63a troubeshooting: fix lint errors
2020-12-11 22:44:17 +01:00

460 lines
15 KiB
ReStructuredText
Raw Blame History

.. _troubleshooting:
###############
Troubleshooting
###############
Sometimes things break or don't work as expected. This section describes
several troubleshooting tools provided by VyOS that can help when something
goes wrong.
******************
Connectivity Tests
******************
Basic Connectivity Tests
========================
Verifying connectivity can be done with the familiar `ping` and `traceroute`
commands. The options for each are shown (the options for each command were
displayed using the built-in help as described in the :ref:`cli`
section and are omitted from the output here):
.. opcmd:: ping <destination>
Send ICMP echo requests to destination host. There are multiple options to
ping, inkl. VRF support.
.. code-block:: none
vyos@vyos:~$ ping 10.1.1.1
Possible completions:
<Enter> Execute the current command
adaptive Ping options
allow-broadcast
audible
bypass-route
count
deadline
flood
interface
interval
mark
no-loopback
numeric
pattern
quiet
record-route
size
timestamp
tos
ttl
verbose
vrf
.. opcmd:: traceroute <destination>
Trace path to target.
.. code-block:: none
vyos@vyos:~$ traceroute
Possible completions:
<hostname> Track network path to specified node
<x.x.x.x>
<h:h:h:h:h:h:h:h>
ipv4 Track network path to <hostname|IPv4 address>
ipv6 Track network path to <hostname|IPv6 address>
Advanced Connectivity Tests
===========================
.. opcmd:: monitor traceroute <destination>
However, another helper is available which combines ping and traceroute
into a single tool. An example of its output is shown:
.. code-block:: none
vyos@vyos:~$ mtr 10.62.212.12
My traceroute [v0.85]
vyos (0.0.0.0)
Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev
1. 10.11.110.4 0.0% 34 0.5 0.5 0.4 0.8 0.1
2. 10.62.255.184 0.0% 34 1.1 1.0 0.9 1.4 0.1
3. 10.62.255.71 0.0% 34 1.4 1.4 1.3 2.0 0.1
4. 10.62.212.12 0.0% 34 1.6 1.6 1.6 1.7 0.0
.. note:: The output consumes the screen and will replace your command
prompt.
Several options are available for changing the display output. Press `h` to
invoke the built in help system. To quit, just press `q` and you'll be
returned to the VyOS command prompt.
IPv6 Topology Discovery
=======================
IPv6 uses different techniques to discover its Neighbors/topology.
Router Discovery
----------------
.. opcmd:: force ipv6-rd interface <interface> [address <ipv6-address>]
Discover routers via eth0.
Example:
.. code-block:: none
vyos@vyos:~$ force ipv6-rd interface eth0
Soliciting ff02::2 (ff02::2) on eth0...
Hop limit : 60 ( 0x3c)
Stateful address conf. : No
Stateful other conf. : No
Mobile home agent : No
Router preference : high
Neighbor discovery proxy : No
Router lifetime : 1800 (0x00000708) seconds
Reachable time : unspecified (0x00000000)
Retransmit time : unspecified (0x00000000)
Prefix : 240e:fe:8ca7:ea01::/64
On-link : Yes
Autonomous address conf.: Yes
Valid time : 2592000 (0x00278d00) seconds
Pref. time : 14400 (0x00003840) seconds
Prefix : fc00:470:f1cd:101::/64
On-link : Yes
Autonomous address conf.: Yes
Valid time : 2592000 (0x00278d00) seconds
Pref. time : 14400 (0x00003840) seconds
Recursive DNS server : fc00:470:f1cd::ff00
DNS server lifetime : 600 (0x00000258) seconds
Source link-layer address: 00:98:2B:F8:3F:11
from fe80::298:2bff:fef8:3f11
Neighbor Discovery
------------------
.. opcmd:: force ipv6-nd interface <interface> address <ipv6-address>
Example:
.. code-block:: none
vyos@vyos:~$ force ipv6-nd interface eth0 address fc00:470:f1cd:101::1
Soliciting fc00:470:f1cd:101::1 (fc00:470:f1cd:101::1) on eth0...
Target link-layer address: 00:98:2B:F8:3F:11 from fc00:470:f1cd:101::1
***************
Interface names
***************
If you find the names of your interfaces have changed, this could be because
your MAC addresses have changed.
* For example, you have a VyOS VM with 4 Ethernet interfaces named
eth0, eth1, eth2 and eth3. Then, you migrate your VyOS VM to a different
host and find your interfaces now are eth4, eth5, eth6 and eth7.
One way to fix this issue **taking control of the MAC addresses** is:
Log into VyOS and run this command to display your interface settings.
.. code-block:: none
show interfaces detail
Take note of MAC addresses.
Now, in order to update a MAC address in the configuration, run this command
specifying the interface name and MAC address you want.
.. code-block:: none
set interfaces eth0 hw-id 00:0c:29:da:a4:fe
If it is a VM, go into the settings of the host and set the MAC address to
the settings found in the config.boot file. You can also set the MAC to
static if the host allows so.
* Another example could be when cloning VyOS VMs in GNS3 and you get into the
same issue: interface names have changed.
And **a more generic way to fix it** is just deleting every MAC address at
the configuration file of the cloned machine. They will be correctly
regenerated automatically.
**********
Monitoring
**********
VyOS features several monitoring tools.
.. code-block:: none
vyos@vyos:~$ monitor
Possible completions:
bandwidth Monitor interface bandwidth in real time
bandwidth-test
Initiate or wait for bandwidth test
cluster Monitor clustering service
command Monitor an operational mode command (refreshes every 2 seconds)
conntrack-sync
Monitor conntrack-sync
content-inspection
Monitor Content-Inspection
dhcp Monitor Dynamic Host Control Protocol (DHCP)
dns Monitor a Domain Name Service (DNS) daemon
firewall Monitor Firewall
https Monitor the Secure Hypertext Transfer Protocol (HTTPS) service
lldp Monitor Link Layer Discovery Protocol (LLDP) daemon
log Monitor last lines of messages file
nat Monitor network address translation (NAT)
ndp Monitor the NDP information received by the router through the device
openvpn Monitor OpenVPN
protocol Monitor routing protocols
snmp Monitor Simple Network Management Protocol (SNMP) daemon
stop-all Stop all current background monitoring processes
traceroute Monitor the path to a destination in realtime
traffic Monitor traffic dumps
vpn Monitor VPN
vrrp Monitor Virtual Router Redundancy Protocol (VRRP)
webproxy Monitor Webproxy service
Traffic Dumps
=============
To monitor interface traffic, issue the :code:`monitor traffic interface <name>`
command, replacing `<name>` with your chosen interface.
.. code-block:: none
vyos@vyos:~$ monitor traffic interface eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:54:28.581601 IP 192.168.0.1 > vyos: ICMP echo request, id 1870, seq 3848, length 64
15:54:28.581660 IP vyos > 192.168.0.1: ICMP echo reply, id 1870, seq 3848, length 64
15:54:29.583399 IP 192.168.0.1 > vyos: ICMP echo request, id 1870, seq 3849, length 64
15:54:29.583454 IP vyos > 192.168.0.1: ICMP echo reply, id 1870, seq 3849, length 64
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel
vyos@vyos:~$
To quit monitoring, press `Ctrl-c` and you'll be returned to the VyOS command
prompt.
Traffic can be filtered and saved.
.. code-block:: none
vyos@vyos:~$ monitor traffic interface eth0
Possible completions:
<Enter> Execute the current command
filter Monitor traffic matching filter conditions
save Save traffic dump from an interface to a file
Interface Bandwidth Usage
=========================
to take a quick view on the used bandwidth of an interface use the ``monitor
bandwidth`` command
.. code-block:: none
vyos@vyos:~$ monitor bandwidth interface eth0
show the following:
.. code-block:: none
B (RX Bytes/second)
198.00 .|....|.....................................................
165.00 .|....|.....................................................
132.00 ||..|.|.....................................................
99.00 ||..|.|.....................................................
66.00 |||||||.....................................................
33.00 |||||||.....................................................
1 5 10 15 20 25 30 35 40 45 50 55 60
KiB (TX Bytes/second)
3.67 ......|.....................................................
3.06 ......|.....................................................
2.45 ......|.....................................................
1.84 ......|.....................................................
1.22 ......|.....................................................
0.61 :::::||.....................................................
1 5 10 15 20 25 30 35 40 45 50 55 60
Interface Performance
=====================
To take a look on the network bandwidth between two nodes, the ``monitor
bandwidth-test`` command is used to run iperf.
.. code-block:: none
vyos@vyos:~$ monitor bandwidth-test
Possible completions:
accept Wait for bandwidth test connections (port TCP/5001)
initiate Initiate a bandwidth test
* The ``accept`` command opens a listening iperf server on TCP Port 5001
* The ``initiate`` command connects to that server to perform the test.
.. code-block:: none
vyos@vyos:~$ monitor bandwidth-test initiate
Possible completions:
<hostname> Initiate a bandwidth test to specified host (port TCP/5001)
<x.x.x.x>
<h:h:h:h:h:h:h:h>
Monitor command
===============
The ``monitor command`` command allows you to repeatedly run a command to view
a continuously refreshed output. The command is run and output every 2 seconds,
allowing you to monitor the output continuously without having to re-run the
command. This can be useful to follow routing adjacency formation.
.. code-block:: none
vyos@router:~$ monitor command "show interfaces"
Will clear the screen and show you the output of ``show interfaces`` every
2 seconds.
.. code-block:: none
Every 2.0s: /opt/vyatta/bin/vyatta-op-cmd-wrapper Sun Mar 26 02:49:46 2019
Codes: S - State, L - Link, u - Up, D - Down, A - Admin Down
Interface IP Address S/L Description
--------- ---------- --- -----------
eth0 192.168.1.1/24 u/u
eth0.5 198.51.100.4/24 u/u WAN
lo 127.0.0.1/8 u/u
::1/128
vti0 172.25.254.2/30 u/u
vti1 172.25.254.9/30 u/u
****************
Terminal/Console
****************
Sometimes you need to clear counters or statistics to troubleshoot better.
To do this use the ``clear`` command in Operational mode.
to clear the console output
.. code-block:: none
vyos@vyos:~$ clear console
to clear interface counters
.. code-block:: none
# clear all interfaces
vyos@vyos:~$ clear interface ethernet counters
# clear specific interface
vyos@vyos:~$ clear interface ehternet eth0 counters
The command follow the same logic as the ``set`` command in configuration mode.
.. code-block:: none
# clear all counters of a interface type
vyos@vyos:~$ clear interface <interface_type> counters
# clear counter of a interface in interface_type
vyos@vyos:~$ clear interface <interface_type> <interace_name> counters
to clear counters on firewall rulesets or single rules
.. code-block:: none
vyos@vyos:~$ clear firewall name <ipv4 ruleset name> counters
vyos@vyos:~$ clear firewall name <ipv4 ruleset name> rule <rule#> counters
vyos@vyos:~$ clear firewall ipv6-name <ipv6 ruleset name> counters
vyos@vyos:~$ clear firewall ipv6-name <ipv6 ruleset name> rule <rule#> counters
******************
System Information
******************
.. _boot-steps:
Boot Steps
==========
VyOS 1.2 uses `Debian Jessie`_ as the base Linux operating system. Jessie was
the first version of Debian that uses systemd_ as the default init system.
These are the boot steps for VyOS 1.2
1. The BIOS loads Grub (or isolinux for the Live CD)
2. Grub then starts the Linux boot and loads the Linux Kernel ``/boot/vmlinuz``
3. Kernel Launches Systemd ``/lib/systemd/systemd``
4. Systemd loads the VyOS service file
``/lib/systemd/system/vyos-router.service``
5. The service file launches the VyOS router init script
``/usr/libexec/vyos/init/vyos-router`` - this is part of the vyatta-cfg_
Debian package
1. Starts FRR_ - successor to `GNU Zebra`_ and Quagga_
2. Initialises the boot configuration file - copies over
``config.boot.default`` if there is no configuration
3. Runs the configuration migration, if the configuration is for an older
version of VyOS
4. Runs The pre-config script, if there is one
``/config/scripts/vyos-preconfig-bootup.script``
5. If the config file was upgraded, runs any post upgrade scripts
``/config/scripts/post-upgrade.d``
6. Starts ``rl-system`` and ``firewall``
7. Mounts the ``/boot`` partition
8. The boot configuration file is then applied by ``/opt/vyatta/sbin/
vyatta-boot-config-loader/opt/vyatta/etc/config/config.boot``
1. The config loader script writes log entries to
``/var/log/vyatta-config-loader.log``
10. Runs ``telinit q`` to tell the init system to reload ``/etc/inittab``
11. Finally it runs the post-config script
``/config/scripts/vyos-postconfig-bootup.script``
.. stop_vyoslinter
.. _Quagga: https://www.quagga.net/
.. _`GNU Zebra`: https://www.gnu.org/software/zebra/
.. _FRR: https://frrouting.org/
.. _vyatta-cfg: https://github.com/vyos/vyatta-cfg
.. _systemd: https://freedesktop.org/wiki/Software/systemd/
.. _`Debian Jessie`: https://www.debian.org/releases/jessie/
.. _tshark: https://www.wireshark.org/docs/man-pages/tshark.html
.. _`PCAP filter expressions`: http://www.tcpdump.org/manpages/pcap-filter.7.html
.. start_vyoslinter
Reference in New Issue View Git Blame Copy Permalink