mirror of
https://github.com/clearlinux/clear-linux-documentation.git
synced 2026-10-03 15:39:59 +00:00
Revert "Merge of latest changes into RTD theme to enable multi-language support (#604)"
This reverts commit 186f1e02b8.
This commit is contained in:
Kevin Putnam
@@ -0,0 +1,92 @@
|
||||
.. _kubernetes-bp:
|
||||
|
||||
Kubernetes Best Practices on |CL|
|
||||
#################################
|
||||
|
||||
Use swupd to update clusters
|
||||
****************************
|
||||
|
||||
This tutorial shows you how to manage your Kubernetes cluster while using
|
||||
:command:`swupd` to update |CL-ATTR|.
|
||||
|
||||
In our tutorial :ref:`kubernetes`, we explain how to set up a Kubernetes
|
||||
cluster on |CL| using `kubeadm`. `Kubeadm documentation`_ often builds on the
|
||||
assumption that the distribution uses a traditional package manager (e.g.,
|
||||
RPM/DEB).
|
||||
|
||||
In contrast, |CL| uses `swupd` to update the OS, which in this case updates
|
||||
all of the kubernetes node and client binaries simultaneously, as part of
|
||||
the `cloud-native-basic` bundle (e.g., kubectl, kubeadm, kubelet). Running
|
||||
:command:`sudo swupd update` requires special care to ensure the OS
|
||||
incorporates the latest Kubernetes upgrades.
|
||||
|
||||
This document describes best practices to manage cluster upgrades with
|
||||
`kubeadm` on a |CL|-based cluster.
|
||||
|
||||
Prerequisites
|
||||
*************
|
||||
|
||||
Assure that you:
|
||||
|
||||
* Completed :ref:`kubernetes`
|
||||
* Installed the bundle `cloud-native-basic`
|
||||
|
||||
.. note::
|
||||
|
||||
Other Linux\* distros shown in Kubernetes upgrade documentation reflect
|
||||
`apt-get update`, `apt-mark hold kubeadm`, and similar commands; however, such commands **are not valid** on |CL|.
|
||||
|
||||
Update the control plane
|
||||
************************
|
||||
|
||||
#. Read kubernetes documentation `before you begin`_.
|
||||
|
||||
#. On your master node, run the command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd update
|
||||
|
||||
.. note::
|
||||
|
||||
If the minor version of Kubernetes changes, |CL| shows a message-of-the-day, or `motd`. When the `motd` appears, you **must postpone** a kubelet restart on master and nodes until the control plane is properly updated. :command:`swupd update` does not restart services automatically unless explicitly configured to do so.
|
||||
|
||||
#. Now follow these instructions in kubernetes documentation.
|
||||
|
||||
* `Upgrade control plane`_
|
||||
* `Drain control plane node`_
|
||||
* `Restart Kubelet and undrain node`_
|
||||
|
||||
Update worker nodes
|
||||
*******************
|
||||
|
||||
#. On each worker node, run the command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd update
|
||||
|
||||
#. Now follow these instructions in kubernetes documentation:
|
||||
|
||||
* `Drain node`_
|
||||
* `Update kubelet configuration`_
|
||||
* `Restart Kubelet and undrain node`_
|
||||
|
||||
.. _Kubeadm documentation: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-upgrade/
|
||||
|
||||
.. _Restart Kubelet and undrain node: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/#restart-the-kubelet-for-all-nodes
|
||||
|
||||
.. _Update kubelet configuration: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/#upgrade-the-kubelet-config-on-worker-nodes
|
||||
|
||||
.. _Drain node: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/#drain-control-plane-and-worker-nodes
|
||||
|
||||
.. _Restart kubelet and undrain node: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/#restart-the-kubelet-for-all-nodes
|
||||
|
||||
.. _Upgrade control plane: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/#upgrade-the-control-plane-node
|
||||
|
||||
.. _Drain control plane node: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/#drain-control-plane-and-worker-nodes
|
||||
|
||||
.. _Kubeadmn documentation: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
|
||||
|
||||
.. _before you begin: https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade-1-13/#before-you-begin
|
||||
|
||||
@@ -0,0 +1,414 @@
|
||||
.. _kubernetes:
|
||||
|
||||
Run Kubernetes\*
|
||||
################
|
||||
|
||||
This tutorial describes how to install, configure, and run the
|
||||
`Kubernetes container orchestration system`_ on |CL-ATTR| using CRI+O and
|
||||
kata-runtime.
|
||||
|
||||
Kubernetes\* is an open source system for automating deployment, scaling, and
|
||||
management of containerized applications. It groups containers that make up
|
||||
an application into logical units for easy management and discovery.
|
||||
|
||||
Kata Containers\* kata-runtime adheres to
|
||||
:abbr:`OCI (Open Container Initiative*)` guidelines and work seamlessly with
|
||||
Kubernetes. `Kata Containers`_ provide strong isolation for untrusted
|
||||
workloads or multi-tenant scenarios. Kata Containers can be
|
||||
allocated on a per-pod basis so you can mix and match both on the same host
|
||||
to suit your needs.
|
||||
|
||||
.. contents:: :local:
|
||||
:depth: 1
|
||||
|
||||
Prerequisites
|
||||
*************
|
||||
|
||||
This tutorial assumes you have already installed |CL|. For detailed
|
||||
instructions on installing |CL| on a bare metal system, follow the
|
||||
:ref:`bare metal installation tutorial<bare-metal-install-desktop>`.
|
||||
Learn about the benefits of having an up-to-date system for cloud
|
||||
orchestration on the :ref:`swupd-about` page.
|
||||
|
||||
Before you install any new packages, update |CL| with the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd update
|
||||
|
||||
Install Kubernetes and CRI runtimes
|
||||
***********************************
|
||||
|
||||
Kubernetes, a set of supported :abbr:`CRI (Container Runtime Interface)`
|
||||
runtimes, and networking plugins, are included in the `cloud-native-basic`_
|
||||
bundle.
|
||||
|
||||
To install this framework, enter the following command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd bundle-add cloud-native-basic
|
||||
|
||||
.. note::
|
||||
|
||||
For more on networking plugins, see `Install pod network add-on`_.
|
||||
|
||||
Configure Kubernetes
|
||||
********************
|
||||
|
||||
This tutorial uses the basic default Kubernetes configuration for simplicity.
|
||||
You must define your Kubernetes configuration according to your specific
|
||||
deployment and your security needs.
|
||||
|
||||
#. Enable IP forwarding to avoid kubeadm `preflight check`_ errors:
|
||||
|
||||
Create (or edit if it exists) the file :file:`/etc/sysctl.d/60-k8s.conf`
|
||||
and include the following line:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
net.ipv4.ip_forward = 1
|
||||
|
||||
Apply the change:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl restart systemd-sysctl
|
||||
|
||||
#. Enable the kubelet service:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl enable kubelet.service
|
||||
|
||||
#. Disable swap using one of the following methods, either:
|
||||
|
||||
a) Temporarily:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swapoff -a
|
||||
|
||||
.. note::
|
||||
|
||||
Swap will be enabled at next reboot, causing failures in
|
||||
your cluster.
|
||||
|
||||
or:
|
||||
|
||||
b) Permanently:
|
||||
|
||||
Mask the swap partition:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl mask $(sed -n -e 's#^/dev/\([0-9a-z]*\).*#dev-\1.swap#p' /proc/swaps) 2>/dev/null
|
||||
sudo swapoff -a
|
||||
|
||||
.. note::
|
||||
|
||||
On systems with limited resources, some performance degradation may
|
||||
be observed while swap is disabled.
|
||||
|
||||
#. Switch to root to modify the `hosts` file:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo -s
|
||||
|
||||
#. Create (or edit if it exists) the hosts file that Kubernetes will read to
|
||||
locate the master's host:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
echo "127.0.0.1 localhost `hostname`" >> /etc/hosts
|
||||
|
||||
#. Exit root:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
exit
|
||||
|
||||
Configure and run Kubernetes
|
||||
****************************
|
||||
|
||||
This section describes how to configure and run Kubernetes with CRI-O and kata-runtime.
|
||||
|
||||
Configure and run CRI-O + kata-runtime
|
||||
======================================
|
||||
|
||||
#. Enable the CRI-O service:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl enable crio.service
|
||||
|
||||
#. Enter the commands:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl restart crio
|
||||
|
||||
#. Initialize the master control plane with the command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo kubeadm init --cri-socket=/run/crio/crio.sock
|
||||
|
||||
Install pod network add-on
|
||||
**************************
|
||||
|
||||
You must choose and install a `pod network add-on`_ to allow your pods to
|
||||
communicate. Check whether or not your add-on requires special flags when you
|
||||
initialize the master control plane.
|
||||
|
||||
The CRI-O default plugin_dir is :file:`/opt/cni/bin`. This must be a
|
||||
writable directory because third-party networking add-ons will install
|
||||
themselves there.
|
||||
|
||||
.. note::
|
||||
|
||||
CNI plugins provided by |CL| are installed as part of *cloud-native-basic*
|
||||
in :file:`/usr/libexec/cni/` and are currently *not* found by CRI-O by
|
||||
default. These separate directories are required because `swupd` controls
|
||||
the content of :file:`/usr` and leaves :file:`/opt` unchanged.
|
||||
|
||||
When using third-party network add-ons that rely on those plugins, such as
|
||||
Weave or Flannel do, make them available by creating symlinks:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo mkdir -p /opt/cni/bin
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
for i in /usr/libexec/cni/*; do sudo ln -sf $i /opt/cni/bin; done
|
||||
|
||||
**Notes about Weave Net add-on**
|
||||
|
||||
The Weave Net add-on works by default when the above configuration is done.
|
||||
|
||||
**Notes about flannel add-on**
|
||||
|
||||
If you choose the `flannel` add-on, then you must add the following to the
|
||||
`kubeadm init` command:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
--pod-network-cidr 10.244.0.0/16
|
||||
|
||||
If you are using CRI-O and `flannel` and you want to use Kata Containers,
|
||||
edit the :file:`/etc/crio/crio.conf` file to add:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
[crio.runtime]
|
||||
manage_network_ns_lifecycle = true
|
||||
|
||||
Use your cluster
|
||||
****************
|
||||
|
||||
Once your master control plane is successfully initialized, instructions on
|
||||
how to use your cluster and its *IP*, *token*, and *hash* values are
|
||||
displayed. It is important that you record the cluster values because they
|
||||
are needed when joining worker nodes to the cluster. Some values have a valid
|
||||
period. The values are presented in a format similar to:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
kubeadm join <master-ip>:<master-port> --token <token> --discovery-token-ca-cert-hash <hash>
|
||||
|
||||
**Congratulations!**
|
||||
|
||||
You've successfully installed and set up Kubernetes in |CL| using CRI-O and
|
||||
kata-runtime. You are now ready to follow on-screen instructions to deploy a
|
||||
pod network to the cluster and join worker nodes with the displayed token
|
||||
and IP information.
|
||||
|
||||
Related topics
|
||||
**************
|
||||
|
||||
Read the Kubernetes documentation to learn more about:
|
||||
|
||||
* `Understanding basic Kubernetes architecture`_
|
||||
|
||||
* `Deploying an application to your cluster`_
|
||||
|
||||
* Installing a `pod network add-on`_
|
||||
|
||||
* `Joining your nodes`_
|
||||
|
||||
Cloud native setup automation (optional)
|
||||
****************************************
|
||||
|
||||
Clone the `cloud-native-setup`_ repository on your system and follow the instructions. This repository includes helper scripts to automate configuration.
|
||||
|
||||
Package configuration customization (optional)
|
||||
**********************************************
|
||||
|
||||
|CL| is a stateless system that looks for user-defined package configuration
|
||||
files in the :file:`/etc/<package-name>` directory to be used as default. If
|
||||
user-defined files are not found, |CL| uses the distribution-provided
|
||||
configuration files for each package.
|
||||
|
||||
If you customize any of the default package configuration files, you **must**
|
||||
store the customized files in the :file:`/etc/` directory. If you edit any of
|
||||
the distribution-provided default files, your changes will be lost in the
|
||||
next system update.
|
||||
|
||||
For example, to customize CRI-O configuration in your system, run the
|
||||
following commands:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo mkdir /etc/crio
|
||||
sudo cp /usr/share/defaults/crio/crio.conf /etc/crio/
|
||||
sudo $EDITOR /etc/crio/crio.conf
|
||||
|
||||
Learn more about `Stateless`_ in |CL| and view the |CL| `documentation`_.
|
||||
|
||||
Proxy configuration (optional)
|
||||
******************************
|
||||
|
||||
If you use a proxy server, you must set your proxy environment variables and
|
||||
create an appropriate proxy configuration file for both CRI-O services. Consult your IT department if you are behind a corporate proxy for
|
||||
the appropriate values. Ensure that your local IP is **explicitly included**
|
||||
in the environment variable *NO_PROXY*. (Setting *localhost* is not enough.)
|
||||
|
||||
If you have already set your proxy environment variables, run the following
|
||||
commands as a shell script to configure all of these services in one step:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
services=('crio')
|
||||
for s in "${services[@]}"; do
|
||||
sudo mkdir -p "/etc/systemd/system/${s}.service.d/"
|
||||
cat << EOF | sudo tee "/etc/systemd/system/${s}.service.d/proxy.conf"
|
||||
[Service]
|
||||
Environment="HTTP_PROXY=${http_proxy}"
|
||||
Environment="HTTPS_PROXY=${https_proxy}"
|
||||
Environment="SOCKS_PROXY=${socks_proxy}"
|
||||
Environment="NO_PROXY=${no_proxy}"
|
||||
EOF
|
||||
done
|
||||
|
||||
Next steps
|
||||
**********
|
||||
|
||||
:ref:`kubernetes-bp`
|
||||
|
||||
|
||||
Troubleshooting
|
||||
***************
|
||||
|
||||
* <HOSTNAME> not found in <IP> message.
|
||||
|
||||
Your DNS server may not be appropriately configured. Try adding an
|
||||
entry to the :file:`/etc/hosts` file with your host's IP and Name.
|
||||
|
||||
For example: 100.200.50.20 myhost
|
||||
|
||||
Use the commands :command:`hostname` and :command:`hostname -I` to retrieve them.
|
||||
|
||||
* Images cannot be pulled.
|
||||
|
||||
You may be behind a proxy server. Try configuring your proxy settings,
|
||||
using the environment variables *HTTP_PROXY*, *HTTPS_PROXY*, and *NO_PROXY*
|
||||
as required in your environment.
|
||||
|
||||
* Connection refused error.
|
||||
|
||||
If you are behind a proxy server, you may need to add the master's IP to
|
||||
the environment variable *NO_PROXY*.
|
||||
|
||||
* Connection timed-out or Access Refused errors.
|
||||
|
||||
You must ensure that the appropriate proxy settings are available from the
|
||||
same terminal where you will initialize the control plane. To verify the
|
||||
proxy settings that Kubernetes will actually use, run the commands:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
echo $HTTP_PROXY
|
||||
echo $HTTPS_PROXY
|
||||
echo $NO_PROXY
|
||||
|
||||
If the displayed proxy values are different from your assigned values, the
|
||||
cluster initialization will fail. Contact your IT support team to learn how
|
||||
to set the proxy variables permanently, and how to make them available for
|
||||
all the types of access that you will use, such as remote SSH access.
|
||||
|
||||
If the result of the above commands is blank, you may need to add a
|
||||
``profile`` to the :file:`/etc` directory. To do so, follow these steps.
|
||||
|
||||
#. Create a `profile` in :file:`/etc`
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo touch profile
|
||||
|
||||
#. With a preferred editor, open `profile`, and enter your proxy settings.
|
||||
Example shown below.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export "HTTP_PROXY=http://proxy.example.com:443"
|
||||
export "HTTPS_PROXY=http://proxy.example.com:445"
|
||||
export "SOCKS_PROXY=http://proxy.example.com:1080"
|
||||
export "NO_PROXY= site.com,.site.com,localhost,127.0.0.1,<master IP>
|
||||
|
||||
.. note::
|
||||
|
||||
<master IP> can be obtained by running :command:`ifconfig`.
|
||||
|
||||
#. Save and exit the `profile`.
|
||||
|
||||
#. Run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo source profile
|
||||
|
||||
#. To ensure your system isn't running previous session variables, run:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo kubeadm reset --cri-socket=/run/crio/crio.sock
|
||||
|
||||
#. Continue below while passing `-E` in the command as shown.
|
||||
|
||||
* Missing environment variables.
|
||||
|
||||
If you are behind a proxy server, pass environment variables by adding *-E*
|
||||
to the command that initializes the master control plane.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
/* Kubernetes with CRI-O + kata-runtime */
|
||||
sudo -E kubeadm init --cri-socket=/run/crio/crio.sock
|
||||
|
||||
.. _Kubernetes container orchestration system: https://kubernetes.io/
|
||||
|
||||
.. _Kata Containers: https://katacontainers.io/
|
||||
|
||||
.. _Software Update documentation: https://clearlinux.org/documentation/clear-linux/concepts/swupd-about#updating
|
||||
|
||||
.. _cloud-native-basic: https://github.com/clearlinux/clr-bundles/blob/master/bundles/cloud-native-basic
|
||||
|
||||
.. _preflight check: https://kubernetes.io/docs/reference/setup-tools/kubeadm/implementation-details/#preflight-checks
|
||||
|
||||
.. _Understanding basic Kubernetes architecture: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-3
|
||||
|
||||
.. _Deploying an application to your cluster: https://kubernetes.io/docs/user-journeys/users/application-developer/foundational/#section-2
|
||||
|
||||
.. _pod network add-on: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#pod-network
|
||||
|
||||
.. _Joining your nodes: https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/#join-nodes
|
||||
|
||||
.. _Stateless: https://clearlinux.org/features/stateless
|
||||
|
||||
.. _documentation: https://clearlinux.org/documentation/clear-linux
|
||||
|
||||
.. _cloud-native-setup: https://github.com/clearlinux/cloud-native-setup
|
||||
|
||||
Reference in New Issue
Block a user