vyos/vyos-documentation
1
0
Fork 0
mirror of https://github.com/vyos/vyos-documentation.git synced 2025-11-03 04:12:03 +01:00
Code Issues Packages Projects Releases Wiki Activity

build-vyos: ease build instructions

Browse Source
This commit is contained in:
Christian Poessinger 2020-08-03 15:04:14 +02:00
parent 31744a6ecf
commit 8587946d16
3 changed files with 394 additions and 394 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

629
docs/contributing/build-vyos.rst
View File

@ -1,59 +1,63 @@
.. _build:
Building VyOS
=============
##########
Build VyOS
##########
*************
Prerequisites
*************
There are different ways you can build VyOS.
Building using a :ref:`Docker<build docker>` container, although not the only way, is the
easiest way as all dependencies are managed for you. It also allows you to
build ARM images on a x86 host.
Building using a :ref:`build_docker` container, although not the only way, is the
easiest way as all dependencies are managed for you. However, you can also
set up your own build machine and run a :ref:`build_native`.
However, you can also set up your own build machine and :ref:`build from source<build source>`.
.. note:: Starting with VyOS 1.2 the release model of VyOS has changed. VyOS
is now **free as in speech, but not as in beer**. This means that while
VyOS is still an open source project, the release ISOs are no longer free
and can only be obtained via subscription, or by contributing to the
community.
.. note:: Starting with VyOS 1.2 the release model of VyOS has changed.
VyOS is now **free as in speech, but not as in beer**. This means
that while VyOS is still an open source project, the release ISOs are no
longer free and can only be obtained via subscription, or by contributing to
the community.
The source code remains public and an ISO can be built
using the process outlined here.
The source code remains public and an ISO can be built using the process
outlined in this chapter.
This will guide you though the process of building a VyOS ISO using Docker_.
This process has been tested on clean installs of Debian Jessie, Stretch, and
Buster.
.. _build docker:
.. _build_docker:
Docker
------
======
Installing Docker_ and prerequisites:
.. code-block:: none
$ sudo apt-get update
$ sudo apt-get install -y apt-transport-https ca-certificates curl \
gnupg2 software-properties-common
$ sudo apt-get install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64] \
https://download.docker.com/linux/debian $(lsb_release -cs) stable"
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
$ sudo apt-get update
$ sudo apt-get install -y docker-ce
To be able to use Docker_ without ``sudo``, the current non-root user can be added to the
``docker`` group by calling: ``sudo usermod -aG docker yourusername``
To be able to use Docker_ without ``sudo``, the current non-root user must be
added to the ``docker`` group by calling: ``sudo usermod -aG docker
yourusername``.
.. note:: Doing so grants privileges equivalent to the ``root`` user! It is recommended to remove the non-root user from the ``docker`` group after building the VyOS ISO. See also https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user
.. hint:: Doing so grants privileges equivalent to the ``root`` user! It is
recommended to remove the non-root user from the ``docker`` group after
building the VyOS ISO. See also `Docker as non-root`_.
.. note:: The build process needs to be built on a local file system, building
on SMB or NFS shares will result in the container failing to build properly!
VirtualBox Drive Share is also not an option as block device operations
are not implemented and the drive is always mounted as "nodev"
Build Docker Container
^^^^^^^^^^^^^^^^^^^^^^
Build Container
---------------
The container can built by hand or by fetching the pre-built one from DockerHub.
Using the pre-built containers from the `VyOS DockerHub organisation`_ will
@ -65,6 +69,9 @@ vyos-build repository).
downloaded from DockerHub if it is not found on your local machine when
you build the ISO.
Dockerhub
^^^^^^^^^
To manually download the container from DockerHub, run:
.. code-block:: none
@ -72,134 +79,197 @@ To manually download the container from DockerHub, run:
$ docker pull vyos/vyos-build:crux # For VyOS 1.2
$ docker pull vyos/vyos-build:current # For rolling release
Build from source
^^^^^^^^^^^^^^^^^
The container can also be built directly from source:
.. code-block:: none
$ git clone -b crux --single-branch https://github.com/vyos/vyos-build # For VyOS 1.2
$ git clone -b current --single-branch https://github.com/vyos/vyos-build # For rolling release
# For VyOS 1.2 (crux)
$ git clone -b crux --single-branch https://github.com/vyos/vyos-build
# For VyOS 1.3 (equuleus, current)
$ git clone -b current --single-branch https://github.com/vyos/vyos-build
$ cd vyos-build
$ docker build -t vyos/vyos-build:crux docker # For VyOS 1.2
$ docker build -t vyos/vyos-build docker # For rollign release
.. note:: Since VyOS has switched to Debian (10) Buster in its ``master`` branch,
the that the used is different from the one used for ``crux`` branch. Hence you
will need one separate container for each branch
.. note:: Since VyOS has switched to Debian (10) Buster in its ``current`` branch,
you will require individual container for `current` and `crux` builds.
.. _build_iso:
Tips and Tricks
^^^^^^^^^^^^^^^
Build ISO
^^^^^^^^^
If you have not build your own Docker image, you need to clone the repository to your local machine:
You can create yourself some handy Bash aliases to always launch the latest -
per release train (`current` or `crux`) - container. Add the following to your
``.bash_aliases`` file:
.. code-block:: none
$ git clone -b crux --single-branch https://github.com/vyos/vyos-build # For VyOS 1.2
$ git clone -b current --single-branch https://github.com/vyos/vyos-build # For rolling release
alias vybld='docker pull vyos/vyos-build:current && docker run --rm -it \
-v "$(pwd)":/vyos \
-v "$HOME/.gitconfig":/etc/gitconfig \
-v "$HOME/.bash_aliases":/home/vyos_bld/.bash_aliases \
-v "$HOME/.bashrc":/home/vyos_bld/.bashrc \
-w /vyos --privileged --sysctl net.ipv6.conf.lo.disable_ipv6=0 \
-e GOSU_UID=$(id -u) -e GOSU_GID=$(id -g) \
vyos/vyos-build:current bash'
alias vybld_crux='docker pull vyos/vyos-build:crux && docker run --rm -it \
-v "$(pwd)":/vyos \
-v "$HOME/.gitconfig":/etc/gitconfig \
-v "$HOME/.bash_aliases":/home/vyos_bld/.bash_aliases \
-v "$HOME/.bashrc":/home/vyos_bld/.bashrc \
-w /vyos --privileged --sysctl net.ipv6.conf.lo.disable_ipv6=0 \
-e GOSU_UID=$(id -u) -e GOSU_GID=$(id -g) \
vyos/vyos-build:crux bash'
Now you are prepared with two new aliases ``vybld`` and ``vybld_crux`` to spwan
your development containers in your current working directory.
.. _build_native:
Native Build
============
To build VyOS natively you require a properly configured build host with the
following Debian versions installed:
- Debian Jessie for VyOS 1.2 (crux)
- Debian Buster for VyOS 1.3 (equuleus, current) - aka the rolling release
To start, clone the repository to your local machine:
.. code-block:: none
# For VyOS 1.2 (crux)
$ git clone -b crux --single-branch https://github.com/vyos/vyos-build
# For VyOS 1.3 (equuleus, current)
$ git clone -b current --single-branch https://github.com/vyos/vyos-build
For the packages required, you can refer to the ``docker/Dockerfile`` file
in the repository_. The ``./configure`` script will also warn you if any
dependencies are missing.
Once you have the required dependencies installed, you may proceed with the
steps descirbed in :ref:`build_iso`.
.. _build_iso:
*********
Build ISO
*********
Now as you are aware of the prerequisites we can continue and build our own
ISO from source. For this we have to fetch the latest source code from GitHub.
Please note as this will differ for both `current` and `crux`.
.. code-block:: none
# For VyOS 1.2 (crux)
$ git clone -b crux --single-branch https://github.com/vyos/vyos-build
# For VyOS 1.3 (equuleus, current)
$ git clone -b current --single-branch https://github.com/vyos/vyos-build
Now a fresh build of the VyOS ISO can begin. Change directory to the ``vyos-build`` directory and run:
.. code-block:: none
$ cd vyos-build
$ docker run --rm -it --privileged -v $(pwd)/vyos-build:/vyos -w /vyos vyos/vyos-build:crux bash # For VyOS 1.2
$ docker run --rm -it --privileged -v $(pwd)/vyos-build:/vyos -w /vyos vyos/vyos-build bash # For rolling release
vyos_bld@d4220bb519a0:/vyos# ./configure --architecture amd64 \
--build-by "your@email.tld" \
--build-type release --version 1.2.5
# For VyOS 1.2 (crux)
$ docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build:crux bash
# For VyOS 1.3 (equuleus, current)
$ docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build bash
Start the build:
.. code-block:: none
vyos_bld@d4220bb519a0:/vyos# ./configure --architecture amd64 --build-by "j.randomhacker@vyos.io"
vyos_bld@d4220bb519a0:/vyos# sudo make iso
When the build is successful, the resulting iso can be found inside the ``build``
When the build is successful, the resulting iso can be found inside the ``build``
directory as ``live-image-[architecture].hybrid.iso``.
.. note:: Attempting to use the docker build image on MacOS or Windows will fail
as docker does not expose all the filesystem feature required to the container.
Building within a VirtualBox server on Mac or Windows is however possible.
Good luck!
.. note: Make sure to choose the matching container for the version of VyOS
that is being built, ``vyos/vyos-build:crux`` for VyOS 1.2 (crux) and
``vyos/vyos-build`` for rolling release.
.. hint:: Attempting to use the Docker build image on MacOS will fail as
Docker does not expose all the filesystem feature required to the container.
Building within a VirtualBox server on Mac however possible.
.. hint:: Building VyOS on Windows WSL2 with Docker integrated into WSL2 will work
like a charm. No problems are known so far!
.. _build source:
From source
-----------
To build from source, you will need:
- Debian Buster for VyOS 1.2
- Debian Stretch for the rolling releases
To start, clone the repository to your local machine:
.. code-block:: none
$ git clone -b crux --single-branch https://github.com/vyos/vyos-build # For VyOS 1.2
$ git clone -b current --single-branch https://github.com/vyos/vyos-build # For rolling release
For the packages required, you can refer to the ``docker/Dockerfile`` file
in the repository_. The ``./configure`` script will also warn you if any
dependencies are missing.
Once you have the required dependencies, you may configure the build by
running ``./configure`` with your options. For details, refer to
:ref:`Customizing the build<customize>`.
Once you have configured your build, build the ISO by running:
.. code-block:: none
$ sudo make iso
The successfully built ISO should now be in the ``build/`` directory as
``live-image-[architecture].hybrid.iso``.
.. _customize:
Customizing the build
---------------------
Customize
=========
This ISO can be customized with the following list of configure options.
The full and current list can be generated with ``./configure --help``:
.. code-block:: none
-h, --help show this help message and exit
--architecture ARCHITECTURE
Image target architecture (amd64 or i586 or armhf)
--build-by BUILD_BY Builder identifier (e.g. jrandomhacker@example.net)
--custom-package CUSTOM_PACKAGES
Custom packages to install from repositories
--build-type BUILD_TYPE
Build type, release or development
--debian-security-mirror DEBIAN_SECURITY_MIRROR
Debian security updated mirror
--version VERSION Version number (release builds only)
--debian-mirror DEBIAN_MIRROR
Debian repository mirror for ISO build
--vyos-mirror VYOS_MIRROR
VyOS package mirror
--pbuilder-debian-mirror PBUILDER_DEBIAN_MIRROR
Debian repository mirror for pbuilder env bootstrap
--debug Enable debug output
--custom-apt-entry CUSTOM_APT_ENTRY
Custom APT entry
--custom-apt-key CUSTOM_APT_KEY
Custom APT key file
$ ./configure --help
usage: configure [-h] [--architecture ARCHITECTURE] [--build-by BUILD_BY]
[--debian-mirror DEBIAN_MIRROR]
[--debian-security-mirror DEBIAN_SECURITY_MIRROR]
[--pbuilder-debian-mirror PBUILDER_DEBIAN_MIRROR]
[--vyos-mirror VYOS_MIRROR] [--build-type BUILD_TYPE]
[--version VERSION] [--build-comment BUILD_COMMENT] [--debug]
[--custom-apt-entry CUSTOM_APT_ENTRY]
[--custom-apt-key CUSTOM_APT_KEY]
[--custom-package CUSTOM_PACKAGE]
Customized packages
^^^^^^^^^^^^^^^^^^^
optional arguments:
-h, --help show this help message and exit
--architecture ARCHITECTURE
Image target architecture (amd64 or i386 or armhf)
--build-by BUILD_BY Builder identifier (e.g. jrandomhacker@example.net)
--debian-mirror DEBIAN_MIRROR
Debian repository mirror for ISO build
--debian-security-mirror DEBIAN_SECURITY_MIRROR
Debian security updates mirror
--pbuilder-debian-mirror PBUILDER_DEBIAN_MIRROR
Debian repository mirror for pbuilder env bootstrap
--vyos-mirror VYOS_MIRROR
VyOS package mirror
--build-type BUILD_TYPE
Build type, release or development
--version VERSION Version number (release builds only)
--build-comment BUILD_COMMENT
Optional build comment
--debug Enable debug output
--custom-apt-entry CUSTOM_APT_ENTRY
Custom APT entry
--custom-apt-key CUSTOM_APT_KEY
Custom APT key file
--custom-package CUSTOM_PACKAGE
Custom package to install from repositories
.. _build_custom_packages:
Packages
--------
If you are brave enough to build yourself an ISO image containing any modified
package from our GitHub organisation - this is the place to be.
Building an ISO with a customized package is in no way different then building
a regular (customized or not) ISO image. Simply place your modified `*.deb`
package inside the `packages` folder within `vyos-build`. You may need to create
the folder in advance.
Any "modified" package may refer to an altered version of e.g. vyos-1x package
that you would like to test before filing a PullRequest on GitHub.
Building an ISO with any customized package is in no way different then
building a regular (customized or not) ISO image. Simply place your modified
`*.deb` package inside the `packages` folder within `vyos-build`. The build
process will then pickup your custom package and integrate it into your ISO.
Troubleshooting
---------------
@ -229,307 +299,78 @@ during ISO build.
.. _build_packages:
Build packages
--------------
********
Packages
********
VyOS requires a bunch of packages which are VyOS specific and thus can not be
found in any Debian Upstream mirrror. Those packages can be found at the
`VyOS GitHub project`_ and there is a nice helper script available to build and
list those individual packages.
VyOS itself comes with a bunch of packages which are specific to our system and
thus can not be found in any Debian mirrror. Those packages can be found at the
`VyOS GitHub project`_ in their source format can can easily be compiled into
a custom Debian (*.deb) package.
``scripts/build-packages`` provides an easy interface to automate the process
of building all VyOS related packages that are not part of the upstream Debian
version. Execute it in the root of the ``vyos-build`` directory to start
compilation.
The easiest way to compile your package is with the above mentioned
:ref:`build_docker` container, it includes all required dependencies for
all VyOS related packages.
Assume we want to build the vyos-1x package on our own and modify it to our
needs. We first need to clone the repository from GitHub.
.. code-block:: none
$ git clone https://github.com/vyos/vyos-1x
Build
=====
Launch Docker container and build package
.. code-block:: none
# For VyOS 1.3 (equuleus, current)
$ docker run --rm -it --privileged -v $(pwd):/vyos -w /vyos vyos/vyos-build bash
# Change to source directory
$ cd vyos-1x
# Build DEB
$ dpkg-buildpackage -uc -us -tc -b
After a minute or two you will find the generated DEB packages next to the vyos-1x
source directory:
.. code-block:: none
$ scripts/build-packages -h
usage: build-packages [-h] [-c | -k | -f] [-v] [-l] [-b BUILD [BUILD ...]]
[-p] [--blacklist BLACKLIST [BLACKLIST ...]]
# ls -al ../vyos-1x*.deb
-rw-r--r-- 1 vyos_bld vyos_bld 567420 Aug 3 12:01 ../vyos-1x_1.3dev0-1847-gb6dcb0a8_all.deb
-rw-r--r-- 1 vyos_bld vyos_bld 3808 Aug 3 12:01 ../vyos-1x-vmware_1.3dev0-1847-gb6dcb0a8_amd64.deb
optional arguments:
-h, --help show this help message and exit
-c, --clean Re-clone required Git repositories
-k, --keep Keep modified Git repositories
-f, --fetch Fetch sources only, no build
-v, --verbose Increase logging verbosity for each occurance
-l, --list-packages List all packages to build
-b BUILD [BUILD ...], --build BUILD [BUILD ...]
Whitespace separated list of packages to build
-p, --parallel Build on all CPUs
--blacklist BLACKLIST [BLACKLIST ...]
Do not build/report packages when calling --list
Install
=======
Git repositoriers are automatically fetched and build on demand. If you want to
work offline you can fetch all source code first with the ``-f`` option.
To take your newly created package on a test drive you can simply SCP it to a
running VyOS instance and install the new *.deb package over the current
running one.
The easiest way to compile is with the above mentioned Docker
container, it includes all dependencies for compiling supported packages.
Just install using the following commands:
.. code-block:: none
$ cd vyos-build
$ docker run --rm -it -v $(pwd):/vyos -w /vyos \
--sysctl net.ipv6.conf.lo.disable_ipv6=0 \
vyos-build scripts/build-packages
vyos@vyos:~$ dpkg --install /tmp/vyos-1x_1.3dev0-1847-gb6dcb0a8_all.deb
(Reading database ... 58209 files and directories currently installed.)
Preparing to unpack .../vyos-1x_1.3dev0-1847-gb6dcb0a8_all.deb ...
Unpacking vyos-1x (1.3dev0-1847-gb6dcb0a8) over (1.3dev0-1847-gb6dcb0a8) ...
Setting up vyos-1x (1.3dev0-1847-gb6dcb0a8) ...
Processing triggers for rsyslog (8.1901.0-1) ...
.. note:: ``--sysctl net.ipv6.conf.lo.disable_ipv6=0`` is required to build the
``vyos-strongswan`` package
.. note:: Prior to executing this script you need to create or build the Docker
container and checkout all packages you want to compile.
Alternatively, on your build server run:
.. code-block:: none
$ sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=0
$ cd vyos-build
$ chmod +x scripts/build-packages
$ ./scripts/build-packages
Building single package(s)
^^^^^^^^^^^^^^^^^^^^^^^^^^
To build a single package use the same script as above but specify packages with
``-b``.
If building using Docker:
.. code-block:: none
$ cd vyos-build
$ docker run --rm -it -v $(pwd):/vyos -w /vyos \
--sysctl net.ipv6.conf.lo.disable_ipv6=0 \ # Only needed for `vyos-strongswan`
vyos-build scripts/build-packages -b <package>
.. note:: ``vyos-strongswan`` will only compile on a Linux system, running on
macOS or Windows might result in a unit test deadlock (it never exits).
If building on build server:
.. code-block:: none
$ sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=0 # Only needed for `vyos-strongswan`
$ cd vyos-build
$ chmod +x scripts/build-packages
$ ./scripts/build-packages -b <package>
Building single package(s) from your own repositories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You can also build packages that are not from the default git repositories,
for example from your own forks of the official VyOS repositories.
First create a directory "packages" at the top level of the vyos-build
repository and clone your package into it (creating a subdirectory with the
package contents). Then checkout the correct branch or commit you want to build
before building the package.
Example using ``git@github.com:myname/vyos-1x.git`` repository to build vyos-1x:
.. code-block:: none
$ cd vyos-build
$ mkdir packages
$ cd packages
$ git clone git@github.com:myname/vyos-1x.git
$ cd ..
If building using Docker:
.. code-block:: none
$ docker run --rm -it -v $(pwd):/vyos -w /vyos \
--sysctl net.ipv6.conf.lo.disable_ipv6=0 \ # Only needed for `vyos-strongswan`
vyos-build scripts/build-packages -b vyos-1x
If building on build server:
.. code-block:: none
$ sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=0 # Only needed for `vyos-strongswan`
$ ./scripts/build-packages -b vyos-1x
.. note:: You need to git pull manually after you commit to the remote and
before rebuilding, the local repository won't be updated automatically.
You can also place the generated *.deb into your ISO build environment to
include it in a custom iso, see :ref:`build_custom_packages` for more
information.
.. warning:: Any packages in the packages directory will be added to the iso
during build, replacing the upstream ones. Make sure you delete them (both
the source directories and built deb packages) if you want to build an iso
from purely upstream packages.
.. _upstream_packages:
Upstream packages
-----------------
Many base system packages are pulled straight from Debian's main and contrib
repositories, but there are exceptions.
This chapter lists those exceptions and gives you a brief overview what we
have done on those packages. If you only want to build yourself a fresh ISO
you can completely skip this chapter. It may become interesting once you have
a VyOS deep dive.
vyos-netplug
^^^^^^^^^^^^
Due to issues in the upstream version that sometimes set interfaces down, a
modified version is used.
The source is located at https://github.com/vyos/vyos-netplug
In the future, we may switch to using systemd infrastructure instead. Building
it doesn't require a special procedure.
keepalived
^^^^^^^^^^
Keepalived normally isn't updated to newer feature releases between Debian
versions, so we are building it from source.
Debian does keep their package in git, but it's upstream tarball imported into
git without its original commit history. To be able to merge new tags in, we
keep a fork of the upstream repository with packaging files imported from
Debian at https://github.com/vyos/keepalived-upstream
strongswan
^^^^^^^^^^
Our StrongSWAN build differs from the upstream:
- strongswan-nm package build is disabled since we don't use NetworkManager
- Patches for DMVPN are merged in
The source is at https://github.com/vyos/vyos-strongswan
DMVPN patches are added by this commit:
https://github.com/vyos/vyos-strongswan/commit/1cf12b0f2f921bfc51affa3b81226
Our op mode scripts use the python-vici module, which is not included in
Debian's build, and isn't quite easy to integrate in that build. For this
reason we debianize that module by hand now, using this procedure:
0. Install https://pypi.org/project/stdeb/
1. `cd vyos-strongswan`
2. `./configure --enable-python-eggs`
3. `cd src/libcharon/plugins/vici/python`
4. `make`
5. `python3 setup.py --command-packages=stdeb.command bdist_deb`
The package ends up in deb_dist dir.
ppp
^^^
Properly renaming PPTP and L2TP interfaces to pptpX and l2tpX from generic and
non-informative pppX requires a patch that is neither in the upstream nor in
Debian.
We keep a fork of Debian's repo at https://github.com/vyos/ppp-debian
The patches for pre-up renaming are:
* https://github.com/vyos/ppp-debian/commit/e728180026a051d2a96396276e7e4ae
* https://github.com/vyos/ppp-debian/commit/f29ba8d9ebb043335a096d70bcd07e9
Additionally, there's a patch for reopening the log file to better support
logging to files, even though it's less essential:
https://github.com/vyos/ppp-debian/commit/dd2ebd5cdcddb40230dc4cc43d374055f
The patches were written by Stephen Hemminger back in the Vyatta times.
mdns-repeater
^^^^^^^^^^^^^
This package doesn't exist in Debian. A debianized fork is kept at
https://github.com/vyos/mdns-repeater
No special build procedure is required.
udp-broadcast-relay
^^^^^^^^^^^^^^^^^^^
This package doesn't exist in Debian. A debianized fork is kept at
https://github.com/vyos/udp-broadcast-relay
No special build procedure is required.
Linux kernel
^^^^^^^^^^^^
In the past a fork of the Kernel source code was kept at the well-known
location of https://github.com/vyos/vyos-kernel - where it is kept for history.
Nowadays the Kernel we use is the upstream source code which is patched
with two additional patches from the good old Vyatta times which never made it
into the mainstream Kernel. The patches can be found here:
https://github.com/vyos/vyos-build-kernel/tree/master/patches/kernel and are
automatically applied to the Kernel by the Jenkins Pipeline which is used to
generate the Kernel binaries.
The Pipeline script not only builds the Kernel with the configuration named
``x86_64_vyos_defconfig`` which is located in the vyos-build-kernel repository,
too - but in addition also builds some Intel out-of-tree drivers, WireGuard
(as long it is not upstreamed) and Accel-PPP.
The ``Jenkinsfile`` tries to be as verbose as possible on each individual build
step.
Linux Firmware
^^^^^^^^^^^^^^
More and more hardware cards require an additional firmware which is not open
source. The Kernel community hosts a special linux-firmware Git repository
with all available binary files which can be loaded by the Kernel.
The ``vyos-build`` repository fetches a specific commit of the linux-firmware
repository and embeds those binaries into the resulting ISO image. This step is
done in the ``data/live-build-config/hooks/live/40-linux-firmware.chroot`` file.
If the firmware needs to be updated it is sufficient to just exchange the Git
commit id we reference in our build.
Intel NIC drivers
^^^^^^^^^^^^^^^^^
We do not make use of the building Intel NIC drivers except for e1000e. Main
reason is that the out of tree Intel drivers seem be perform a bit better,
e.q. have proper receive-side-scaling and multi-queue support.
Drivers are build as part of the Kernel Pipeline - read above.
Accel-PPP
^^^^^^^^^
Accel-PPP used to be an upstream fork for quite some time but now has been
converted to make use of the upstream source code and build system.
It is build as part of the Kernel Pipeline - read above.
hvinfo
^^^^^^
A fork with packaging changes for VyOS is kept at https://github.com/vyos/hvinfo
The original repo is at https://github.com/dmbaturin/hvinfo
It's an Ada program and requires GNAT and gprbuild for building, dependencies
are properly specified so just follow debuild's suggestions.
Per-file modifications
^^^^^^^^^^^^^^^^^^^^^^
vyos-replace package replaces the upstream dhclient-script with a modified
version that is aware of the VyOS config.
.. _Docker: https://www.docker.com
.. _`Docker as non-root`: https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user
.. _VyOS DockerHub organisation: https://hub.docker.com/u/vyos
.. _repository: https://github.com/vyos/vyos-build
.. _VyOS GitHub project: https://github.com/vyos

158
docs/contributing/upstream-packages.rst Normal file
View File

@ -0,0 +1,158 @@
.. _upstream_packages:
Upstream packages
-----------------
Many base system packages are pulled straight from Debian's main and contrib
repositories, but there are exceptions.
This chapter lists those exceptions and gives you a brief overview what we
have done on those packages. If you only want to build yourself a fresh ISO
you can completely skip this chapter. It may become interesting once you have
a VyOS deep dive.
vyos-netplug
^^^^^^^^^^^^
Due to issues in the upstream version that sometimes set interfaces down, a
modified version is used.
The source is located at https://github.com/vyos/vyos-netplug
In the future, we may switch to using systemd infrastructure instead. Building
it doesn't require a special procedure.
keepalived
^^^^^^^^^^
Keepalived normally isn't updated to newer feature releases between Debian
versions, so we are building it from source.
Debian does keep their package in git, but it's upstream tarball imported into
git without its original commit history. To be able to merge new tags in, we
keep a fork of the upstream repository with packaging files imported from
Debian at https://github.com/vyos/keepalived-upstream
strongswan
^^^^^^^^^^
Our StrongSWAN build differs from the upstream:
- strongswan-nm package build is disabled since we don't use NetworkManager
- Patches for DMVPN are merged in
The source is at https://github.com/vyos/vyos-strongswan
DMVPN patches are added by this commit:
https://github.com/vyos/vyos-strongswan/commit/1cf12b0f2f921bfc51affa3b81226
Our op mode scripts use the python-vici module, which is not included in
Debian's build, and isn't quite easy to integrate in that build. For this
reason we debianize that module by hand now, using this procedure:
0. Install https://pypi.org/project/stdeb/
1. `cd vyos-strongswan`
2. `./configure --enable-python-eggs`
3. `cd src/libcharon/plugins/vici/python`
4. `make`
5. `python3 setup.py --command-packages=stdeb.command bdist_deb`
The package ends up in deb_dist dir.
ppp
^^^
Properly renaming PPTP and L2TP interfaces to pptpX and l2tpX from generic and
non-informative pppX requires a patch that is neither in the upstream nor in
Debian.
We keep a fork of Debian's repo at https://github.com/vyos/ppp-debian
The patches for pre-up renaming are:
* https://github.com/vyos/ppp-debian/commit/e728180026a051d2a96396276e7e4ae
* https://github.com/vyos/ppp-debian/commit/f29ba8d9ebb043335a096d70bcd07e9
Additionally, there's a patch for reopening the log file to better support
logging to files, even though it's less essential:
https://github.com/vyos/ppp-debian/commit/dd2ebd5cdcddb40230dc4cc43d374055f
The patches were written by Stephen Hemminger back in the Vyatta times.
mdns-repeater
^^^^^^^^^^^^^
This package doesn't exist in Debian. A debianized fork is kept at
https://github.com/vyos/mdns-repeater
No special build procedure is required.
udp-broadcast-relay
^^^^^^^^^^^^^^^^^^^
This package doesn't exist in Debian. A debianized fork is kept at
https://github.com/vyos/udp-broadcast-relay
No special build procedure is required.
Linux kernel
^^^^^^^^^^^^
In the past a fork of the Kernel source code was kept at the well-known
location of https://github.com/vyos/vyos-kernel - where it is kept for history.
Nowadays the Kernel we use is the upstream source code which is patched
with two additional patches from the good old Vyatta times which never made it
into the mainstream Kernel. The patches can be found here:
https://github.com/vyos/vyos-build-kernel/tree/master/patches/kernel and are
automatically applied to the Kernel by the Jenkins Pipeline which is used to
generate the Kernel binaries.
The Pipeline script not only builds the Kernel with the configuration named
``x86_64_vyos_defconfig`` which is located in the vyos-build-kernel repository,
too - but in addition also builds some Intel out-of-tree drivers, WireGuard
(as long it is not upstreamed) and Accel-PPP.
The ``Jenkinsfile`` tries to be as verbose as possible on each individual build
step.
Linux Firmware
^^^^^^^^^^^^^^
More and more hardware cards require an additional firmware which is not open
source. The Kernel community hosts a special linux-firmware Git repository
with all available binary files which can be loaded by the Kernel.
The ``vyos-build`` repository fetches a specific commit of the linux-firmware
repository and embeds those binaries into the resulting ISO image. This step is
done in the ``data/live-build-config/hooks/live/40-linux-firmware.chroot`` file.
If the firmware needs to be updated it is sufficient to just exchange the Git
commit id we reference in our build.
Intel NIC drivers
^^^^^^^^^^^^^^^^^
We do not make use of the building Intel NIC drivers except for e1000e. Main
reason is that the out of tree Intel drivers seem be perform a bit better,
e.q. have proper receive-side-scaling and multi-queue support.
Drivers are build as part of the Kernel Pipeline - read above.
Accel-PPP
^^^^^^^^^
Accel-PPP used to be an upstream fork for quite some time but now has been
converted to make use of the upstream source code and build system.
It is build as part of the Kernel Pipeline - read above.
hvinfo
^^^^^^
A fork with packaging changes for VyOS is kept at https://github.com/vyos/hvinfo
The original repo is at https://github.com/dmbaturin/hvinfo
It's an Ada program and requires GNAT and gprbuild for building, dependencies
are properly specified so just follow debuild's suggestions.

1
docs/index.rst
View File

@ -72,6 +72,7 @@ VyOS User Guide
:maxdepth: 2
contributing/build-vyos
contributing/upstream-packages
contributing/issues-features
contributing/development
contributing/documentation