Merge branch 'master' into smb

This commit is contained in:
Rodrigo Caballero
2018-04-09 10:50:40 -05:00
committed by GitHub
63 changed files with 3261 additions and 957 deletions
+184
View File
@@ -0,0 +1,184 @@
.. _cl-restart:
Restart system services after an OS update
##########################################
The software life cycle describes how software is created, developed, and
deployed, and includes how to replace or update software. A good OS
provides tools for the entire software life cycle. These tools must include
ways to remove software components properly when replaced with something else.
Most of the work on software update code in |CL| was focused on adding new
software to the system. We recommended that users reboot their system once in
a while, but we did not provide any tools to restart services easily, until
now.
User challenges
***************
It is difficult to determine which services to restart. You can either
evaluate each system and reboot manually, or figure out which services to
restart based on documentation like the |CL| release notes. Since neither
option solves the issue completely, the |CL| team created a solution.
Over the years, several OSes approached the problem and created partial
solutions such as the following:
* Automatically restart services during an upgrade.
* Evaluate services using these steps:
* Mark updates requiring a reboot, such as kernel updates.
* Inform the user of those updates.
* Ask the user to restart the OS.
Both solutions are acceptable for many OSes. However, |CL| updates software
automatically and users do not see notices from the updater unless they review
the journal. |CL| requires a completely different solution, with the following
requirements:
* Eliminate the guesswork about what to restart and under what circumstances.
* Cannot restart everything. Many service daemons do not support an automatic
background restart.
* Fit into the |CL| architectural perspective: be small, quick, and lean.
clr-service-restart functionality
*********************************
Typical reasons to restart a service daemon include:
* A new version replaces the executable file itself.
* A new version replaces a library component used by a service daemon.
Our method restarts daemons when it is really needed, especially
in the case of security updates. The tool restarts daemons by reading
various files in the :file:`procfs` filesystem provided by the kernel.
The second part of the problem is to determine whether or not running
processes are part of a system service. The tool focuses on system services
because most system services are background tasks with no direct user
interaction. Fortunately, :command:`systemd` provides a simple way to:
* Determine which active tasks are within the system domain.
* Determine which tasks map to which service.
We combined both solutions into a low-overhead tool that shows which system
daemons require a restart, as shown below:
Figure 1: Invoke :command:`clr-service-restart`.
.. code-block:: bash
sudo clr-service-restart -a -n
.. code-block:: console
upower.service: needs a restart (a library dependency was updated)
/usr/bin/systemctl --no-ask-password try-restart upower.service
NetworkManager.service: needs a restart (a library dependency was
updated)
/usr/bin/systemctl --no-ask-password try-restart NetworkManager.service
....
:command:`clr-service-restart` implements a whitelist to identify which
daemons can be restarted. The system administrator can customize the default
|CL| OS whitelist using :option:`allow` or :option:`disallow` options for
restarting system services. When a software update occurs,
:command:`clr-service-restart` consults the whitelist to see if a service
daemon is allowed to be restarted or not. See the options section for
details.
Options for clr-service-restart
*******************************
The :option:`allow` option identifies a daemon to restart after an OS software
update. The :command:`clr-service-restart` daemon creates a symlink in
:file:`/etc/clr-service-restart` as a record. The example below tells
:command:`clr-service-restart` to restart the :option:`tallow` daemon after an
OS software update.
.. code-block:: bash
sudo clr-service-restart allow tallow.service
The :option:`disallow` option tells :command:`clr-service-restart` not to
restart the specified daemon even if the OS defaults permit the daemon to be
restarted. The :command:`clr-service-restart` daemon creates a symlink in
:file:`/etc/clr-service-restart` that points to :file:`/dev/null` as a record.
The example below tells :command:`clr-service-restart` not to restart the
:option:`rngd` daemon after an OS software update.
.. code-block:: bash
sudo clr-service-restart disallow rngd
The :option:`default` option makes :command:`clr-service-restart` revert back
to the OS defaults and delete any symlink in :file:`/etc/clr-service-restart`.
The example below tells :command:`clr-service-restart` to restart
:option:`rngd` automatically again, because :option:`rngd` is whitelisted for
automatic service restarts by default in |CL|.
.. code-block:: bash
sudo clr-service-restart default rngd
Monitor options for clr-service-restart
=======================================
:command:`clr-service-restart` works in the background and is invoked with
:command:`swupd` automatically. Review the journal output to verify that
services are restarted after an OS software update.
To monitor :command:`clr-service-restart`, use one or both options described
below.
:option:`-n`
This option makes :command:`clr-service-restart` perform no restarts. Instead
it displays the services that could potentially be restarted. When used,
:command:`clr-service-restart` outputs a list of messages showing:
* Which service needs a restart.
* What unit it is.
* Why it needs a restart.
* Which command is required to restart the unit.
:option:`-a`
This option makes :command:`clr-service-restart` consider all system services,
not only the ones that are whitelisted. Because the default whitelist in |CL|
is relatively short, you can use this option to restart all impacted services
when you log in on the system.
If you pass both options (:option:`-a` and :option:`-n`),
:command:`clr-service-restart` displays a complete list of system services
that require a restart. Use both options to verify that all desired daemons
are restarted.
Telemetry
*********
:command:`clr-service-restart` may cause problems such as a short service
outage when a daemon is being restarted, or if a daemon fails to properly
restart. To minimize issues, :command:`clr-service-restart` creates a
telemetry record and sends it to the optional |CL| telemetry service if both
conditions below are met:
* If a unit fails to automatically restart after an OS update.
* If that unit resides in the system location :file:`/usr/lib/systemd/system`.
If you do not install the |CL| telemetrics bundle, the data is discarded. If
you install the telemetrics bundle and you opt to send telemetry, then the
system unit name is sent to the |CL| telemetry service. We evaluate the report
and update the whitelist to remove services that are not safe to restart.
Conclusion
**********
The |CL| team enjoys coming up with simple and efficient solutions to make
your work easier. We made a github project of :command:`clr-service-restart`
and we invite you to look at the code, share your thoughts, and work with us
on improving the project. You can find the project at:
https://github.com/clearlinux/clr-service-restart
Binary file not shown.

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 65 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 80 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 85 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 91 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 81 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 69 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 47 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 97 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 67 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 58 KiB

@@ -32,6 +32,7 @@ appropriate set of step-by-step instructions to proceed.
virtualbox
vmware-esxi-install-cl
vmware-esxi-preconfigured-cl-image
vmware-player
vmw-player
vmw-player-preconf
hyper-v
@@ -0,0 +1,314 @@
.. _vmw-player-preconf:
Run pre-configured Clear Linux image as a VMware\* Workstation Player guest OS
##############################################################################
`VMware Workstation 14 Player`_ is a type 2 hypervisor. It runs on top of
another operating system such as Windows\* or Linux\*. With VMware ESXi, you
can create, configure, manage, and run |CLOSIA| :abbr:`VMs (Virtual Machines)`
on your local system.
This section shows how to deploy a pre-configured |CL| VMware image on
VMware Workstation 14 Player.
In this tutorial, we perform the following steps:
#. Install the VMware Workstation Player hypervisor
#. Download the latest |CL| pre-configured image
#. VMware image Verify the integrity of the |CL| image
#. Uncompress the |CL| image
#. Create and configure a new VM
#. Attach the pre-configured VMware |CL| image
#. Enable EFI boot support
#. Power on the VM
.. note::
The screenshots on this document show the Windows\* version of the
VMware Workstation 14 Player. The menus and prompts are similar to those
in the Linux version save some minor wording differences.
Install the VMware Workstation Player hypervisor
************************************************
#. Enable :abbr:`Intel® VT (Intel® Virtualization Technology)` and
:abbr:`Intel® VT-d (Intel® Virtualization Technology for Directed I/O)` in
your system's BIOS.
#. `VMware Workstation 14 Player`_ is available for Windows and Linux.
Download your preferred version.
#. Depending on which OS you're running, install it by following one of these
instructions:
* On supported Linux distros:
#. Enable a GUI desktop.
#. Start a terminal emulator.
#. Start the installer by issuing the command below and follow the guided
steps.
.. code-block:: console
sudo sh ./VMware-Player-[version number].x86_64.bundle
* On Windows:
#. Start the installer.
#. Follow the setup wizard.
For additional help, see the `VMware Workstation Player guide`_.
Clear Linux image types
***********************
.. include:: ../../guides/maintenance/image-types.rst
:Start-after: image-types-content:
Download the latest Clear Linux VMware image
********************************************
Get the latest |CL| VMware image from the `image`_ repository.
Look for :file:`clear-[version number]-vmware.vmdk.xz`.
.. include:: ../../guides/maintenance/download-verify-uncompress-windows.rst
:Start-after: verify-windows:
We also provide instructions for other operating systems:
* :ref:`download-verify-uncompress-linux`
* :ref:`download-verify-uncompress-mac`
Create and configure a new VM
*****************************
#. Start the `VMware Workstation Player` app.
#. On the home screen, click :guilabel:`Create a New Virtual Machine`. See
figure 1.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-01.png
:scale: 100%
:alt: VMware Workstation 14 Player - Create a new virtual machine
Figure 1: VMware Workstation 14 Player - Create a new virtual machine
#. On the :guilabel:`Welcome to the New Virtual Machine Wizard` screen, select
the :guilabel:`I will install the operating system later` option.
See figure 2.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-02.png
:scale: 100%
:alt: VMware Workstation 14 Player - Select install operating system
Figure 2: VMware Workstation 14 Player - Select install operating system
later.
#. Click the :guilabel:`Next` button.
#. On the :guilabel:`Select a Guest Operating System` screen, set the
:guilabel:`Guest operating system` setting to :guilabel:`Linux`.
See figure 3.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-03.png
:scale: 100%
:alt: VMware Workstation 14 Player - Select guest operating system type
Figure 3: VMware Workstation 14 Player - Select guest operating system
type
#. Set :guilabel:`Version` setting to
:guilabel:`Other Linux 3.x or later kernel 64-bit`.
#. Click the :guilabel:`Next` button.
#. On the :guilabel:`Name the Virtual Machine` screen, give your new VM a name.
See figure 4.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-04.png
:scale: 100%
:alt: VMware Workstation 14 Player - Name virtual machine
Figure 4: VMware Workstation 14 Player - Name virtual machine
#. Click the :guilabel:`Next` button.
#. On the :guilabel:`Specify Disk Capacity` screen, click the :guilabel:`Next`
button. Keep the default disk settings unchanged. When we attach the pre-configured |CL|
VMware image, we will remove the default virtual disk and replace it with the
pre-configured one. See figure 5.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-05.png
:scale: 100%
:alt: VMware Workstation 14 Player - Set disk capacity
Figure 5: VMware Workstation 14 Player - Set disk capacity
#. On the :guilabel:`Ready to Create Virtual Machine` screen, click the
:guilabel:`Customize Hardware...` button. See figure 6.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-06.png
:scale: 100%
:alt: VMware Workstation 14 Player - Customize hardware
Figure 6: VMware Workstation 14 Player - Customize hardware
#. Under the :guilabel:`Device` list, select :guilabel:`Processors`. See
figure 7.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-07.png
:scale: 100%
:alt: VMware Workstation 14 Player - Set virtualization engine option
Figure 7: VMware Workstation 14 Player - Set virtualization engine option
#. Under the :guilabel:`Virtualization engine` section,
check :guilabel:`Virtualize Intel VT-x/EPT or AMD-V/RVI`.
#. To disconnect the virtual CD/DVD (IDE) since it is not needed, under the
:guilabel:`Device` list, select :guilabel:`New CD/DVD (IDE)`. See figure 8.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-08.png
:scale: 100%
:alt: VMware Workstation 14 Player - Disconnect CD/DVD (IDE)
Figure 8: VMware Workstation 14 Player - Disconnect CD/DVD (IDE)
#. Under the :guilabel:`Device status` section, uncheck
:guilabel:`Connect at power on`.
#. Click the :guilabel:`Close` button.
#. Click the :guilabel:`Finish` button.
Attach the pre-configured Clear Linux VMware image
**************************************************
#. Move the downloaded and uncompressed pre-configured |CL| VMware image file
:file:`clear-[version number]-basic.vmdk` to the directory where your
newly-created VM resides.
.. note::
Depending on the OS, you can typically find the VMware VM files under:
* On Linux distros: :file:`/home/username/vmware`
* On Windows: :file:`C:\Users\username\Documents\Virtual Machines`
#. On the :guilabel:`VMware Workstation Player` home screen, select your
newly-created VM. See figure 9.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-09.png
:scale: 100%
:alt: VMware Workstation 14 Player - Edit virtual machine settings
Figure 9: VMware Workstation 14 Player - Edit virtual machine settings
#. Click :guilabel:`Edit virtual machine settings`.
#. To remove the default hard disk, under the :guilabel:`Device` list, select
:guilabel:`Hard Disk (SCSI)`. See figure 10.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-10.png
:scale: 100%
:alt: VMware Workstation 14 Player - Remove hard drive
Figure 10: VMware Workstation 14 Player - Remove hard drive
#. Click the :guilabel:`Remove` button.
#. To add a new hard disk and attach the pre-configured |CL| VMware image,
click the :guilabel:`Add...` button. See Figure 11.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-11.png
:scale: 100%
:alt: VMware Workstation 14 Player - Add new hard drive
Figure 11: VMware Workstation 14 Player - Add new hard drive
#. Under the :guilabel:`Hardware types` section, select :guilabel:`Hard Disk`.
#. Click the :guilabel:`Next` button.
#. Select your preferred :guilabel:`Virtual disk type`. See figure 12.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-12.png
:scale: 100%
:alt: VMware Workstation 14 Player - Select virtual disk type
Figure 12: VMware Workstation 14 Player - Select virtual disk type
#. Select the :guilabel:`Use an existing virtual disk` option. See figure 13.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-13.png
:scale: 100%
:alt: VMware Workstation 14 Player - Use existing virtual disk
Figure 13: VMware Workstation 14 Player - Use existing virtual disk
#. Click the :guilabel:`Browse` button and select the pre-configured |CL|
VMware image file. See figure 14.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-14.png
:scale: 100%
:alt: VMware Workstation 14 Player - Select ready-made VMware |CL|
Figure 14: VMware Workstation 14 Player - Select ready-made VMware |CL|
image file
#. Click the :guilabel:`Finish` button.
.. note::
When asked to convert the existing virtual disk to a newer format,
selecting either option works.
Enable UEFI boot support
************************
|CL| needs UEFI support to boot.To enable it, add the
following line to the end of your VM's :file:`.vmx` file:
.. code-block:: console
firmware = "efi"
.. note::
Depending on the OS, you can typically find the VMware VM files under:
* On Linux distros: :file:`/home/username/vmware`
* On Windows: :file:`C:\\Users\\username\\Documents\\Virtual Machines`
Power on the VM
***************
After configuring the settings above, power on your |CL| virtual machine.
#. On the :guilabel:`VMware Workstation Player` home screen, select your
VM. See figure 15.
.. figure:: figures/vmw-player-preconf/vmw-player-preconf-15.png
:scale: 100%
:alt: VMware Workstation 14 Player - Power on virtual machine
Figure 15: VMware Workstation 14 Player - Power on virtual machine
#. Click :guilabel:`Play virtual machine`.
For other guides on using the VMWare Player and ESXi, see:
* :ref:`vmw-player`
* :ref:`vmware-esxi-install-cl`
* :ref:`vmware-esxi-preconfigured-cl-image`
.. _VMware ESXi: https://www.vmware.com/products/esxi-and-esx.html
.. _VMware Workstation 14 Player: https://www.vmware.com/products/workstation-player.html
.. _VMware Workstation Player guide: https://docs.vmware.com/en/VMware-Workstation-Player/index.html
.. _latest: https://download.clearlinux.org/image/
.. _image: https://download.clearlinux.org/image
@@ -0,0 +1,313 @@
.. _vmw-player:
Install Clear Linux as a VMware\* Workstation Player guest OS
#############################################################
`VMware Workstation 14 Player`_ is a type 2 hypervisor. It runs on top of
another operating system such as Windows or Linux. With VMware ESXi, you can
create, configure, manage, and run |CLOSIA| :abbr:`VMs (Virtual Machines)` on
your local system.
This section shows how to create a new VM and install |CL| into it with the
VMware Workstation 14 Player hypervisor. Installing |CL| into a new VM
provides you flexibility when configuring the VM. You can configure the VM's
size, number of partitions, installed bundles, etc.
In this tutorial, we perform the following steps:
#. Install the VMware Workstation Player hypervisor
#. Download the latest |CL| installer ISO
#. Verify the integrity of the |CL| image
#. Uncompress the |CL| image
#. Create and configure a new VM
#. Attach the |CL| installer ISO to the VM
#. Install |CL| into the new VM
#. Detach the |CL| installer ISO from the VM
#. Power off the VM
#. Enable EFI boot support
#. Power on the VM
If you prefer to use a pre-configured |CL| VMware image instead,
see our :ref:`vmw-player-preconf` guide.
VMware offers a type 1 hypervisor called `VMware ESXi`_ designed for the
cloud environment. For information on how to install |CL| as guest OS on
it, see :ref:`vmware-esxi-install-cl`.
.. note::
The screenshots on this document show the Windows\* version of the
VMware Workstation 14 Player. The menus and prompts are similar to those
in the Linux version save some minor wording differences.
Install the VMware Workstation Player hypervisor
************************************************
#. Enable :abbr:`Intel® VT (Intel® Virtualization Technology)` and
:abbr:`Intel® VT-d (Intel® Virtualization Technology for Directed I/O)` in
your system's BIOS.
#. `VMware Workstation 14 Player`_ is available for Windows and Linux.
Download your preferred version.
#. Install VMware Workstation 14 Player following the instructions
appropriate for your system's OS:
* On supported Linux distros:
#. Enable a GUI desktop.
#. Start a terminal emulator.
#. Start the installer by issuing the command below and follow the
guided steps.
.. code-block:: console
$ sudo sh ./VMware-Player-[version number].x86_64.bundle
* On Windows:
#. Start the installer.
#. Follow the setup wizard.
For additional help, see the `VMware Workstation Player guide`_.
Clear Linux image types
***********************
.. include:: ../../guides/maintenance/image-types.rst
:Start-after: image-types-content:
Download the latest Clear Linux installer ISO
*********************************************
Get the latest |CL| installer ISO image from the `image`_ repository.
Look for :file:`clear-[version number]-installer.iso.xz`.
.. include:: ../../guides/maintenance/download-verify-uncompress-windows.rst
:Start-after: verify-windows:
We also provide instructions for other operating systems:
* :ref:`download-verify-uncompress-linux`
* :ref:`download-verify-uncompress-mac`
Create and configure a new VM
*****************************
#. Start the `VMware Workstation Player` app.
#. On the home screen, click :guilabel:`Create a New Virtual Machine`. See
Figure 1.
.. figure:: figures/vmw-player/vmw-player-01.png
:scale: 100%
:alt: VMware Workstation 14 Player - Create a new virtual machine
Figure 1: VMware Workstation 14 Player - Create a new virtual
machine
#. On the :guilabel:`Welcome to the New Virtual Machine Wizard` screen,
select the :guilabel:`Installer disc image file (iso)` option.
See Figure 2.
.. figure:: figures/vmw-player/vmw-player-02.png
:scale: 100%
:alt: VMware Workstation 14 Player - Select |CL| installer ISO
Figure 2: VMware Workstation 14 Player - Select |CL| installer ISO
#. Click the :guilabel:`Browse` button and select the uncompressed |CL|
installer ISO.
#. Click the :guilabel:`Next` button.
#. On the :guilabel:`Select a Guest Operating System`, set the
:guilabel:`Guest operating system` setting to :guilabel:`Linux`. See
Figure 3.
.. figure:: figures/vmw-player/vmw-player-03.png
:scale: 100%
:alt: VMware Workstation 14 Player - Select guest operating system type
Figure 3: VMware Workstation 14 Player - Select guest operating system
type
#. Set the :guilabel:`Version` setting to
:guilabel:`Other Linux 3.x or later kernel 64-bit`.
#. Click the :guilabel:`Next` button.
#. On the :guilabel:`Name the Virtual Machine` screen, name the new VM. See
Figure 4.
.. figure:: figures/vmw-player/vmw-player-04.png
:scale: 100%
:alt: VMware Workstation 14 Player - Name virtual machine
Figure 4: VMware Workstation 14 Player - Name virtual machine
#. Click the :guilabel:`Next` button.
#. On the :guilabel:`Specify Disk Capacity` screen, set the VM's maximum disk
size. See Figure 5.
.. figure:: figures/vmw-player/vmw-player-05.png
:scale: 100%
:alt: VMware Workstation 14 Player - Set disk capacity
Figure 5: VMware Workstation 14 Player - Set disk capacity
.. note::
A minimal |CL| installation can exist on 600MB of drive space.
See :ref:`system-requirements` for more details.
#. Click the :guilabel:`Next` button.
#. On the :guilabel:`Ready to Create Virtual Machine` screen, click the
:guilabel:`Customize Hardware...` button. See Figure 6.
.. figure:: figures/vmw-player/vmw-player-06.png
:scale: 100%
:alt: VMware Workstation 14 Player - Customize hardware
Figure 6: VMware Workstation 14 Player - Customize hardware
#. Select :guilabel:`Memory` and set the size to 2GB. See Figure 7.
.. figure:: figures/vmw-player/vmw-player-07.png
:scale: 100%
:alt: VMware Workstation 14 Player - Set memory size
Figure 7: VMware Workstation 14 Player - Set memory size
.. note::
The |CL| installer ISO needs a minimum of 2GB of RAM.
After completing installation, |CL| can run on as little as
128MB of RAM. Thus, you can reduce the memory size if needed.
See :ref:`system-requirements` for more details.
#. Under the :guilabel:`Device` list, select :guilabel:`Processors`. See
Figure 8.
.. figure:: figures/vmw-player/vmw-player-08.png
:scale: 100%
:alt: VMware Workstation 14 Player - Set virtualization engine option
Figure 8: VMware Workstation 14 Player - Set virtualization engine
option
#. Under the :guilabel:`Virtualization engine` section,
check :guilabel:`Virtualize Intel VT-x/EPT or AMD-V/RVI`.
#. Click the :guilabel:`Close` button.
#. Click the :guilabel:`Finish` button.
Install Clear Linux into the new VM
***********************************
#. Select the newly-created VM and click the :guilabel:`Play virtual machine`
button. See Figure 9.
.. figure:: figures/vmw-player/vmw-player-09.png
:scale: 100%
:alt: VMware Workstation 14 Player - Power on virtual machine
Figure 9: VMware Workstation 14 Player - Power on virtual machine
#. Follow the :ref:`install-on-target` guide to complete the installation of
|CL|.
#. After the installation completes, reboot the VM. This reboot restarts the
|CL| installer.
Detach the |CL| installer ISO from the VM
*****************************************
#. To enable the mouse pointer so you access VMware Workstation Player's
menus, press :kbd:`<CTRL>` + :kbd:`<ALT>` on the keyboard.
#. To disconnect the CD/DVD to stop it from booting the |CL| installer ISO
again, click the :guilabel:`Player` menu. See Figure 10.
.. figure:: figures/vmw-player/vmw-player-10.png
:scale: 100%
:alt: VMware Workstation 14 Player - Edit CD/DVD settings
Figure 10: VMware Workstation 14 Player - Edit CD/DVD settings
#. Go to :menuselection:`Removable Devices-->CD/DVD (IDE)-->Settings`.
#. On the :guilabel:`Device status` section, uncheck the
:guilabel:`Connected` and the :guilabel:`Connect at power on` settings.
See Figure 11.
.. figure:: figures/vmw-player/vmw-player-11.png
:scale: 100%
:alt: VMware Workstation 14 Player - Disconnect CD/DVD
Figure 11: VMware Workstation 14 Player - Disconnect CD/DVD
#. Click the :guilabel:`OK` button.
#. To power off the VM, click the :guilabel:`Player` menu. See Figure 12.
.. figure:: figures/vmw-player/vmw-player-12.png
:scale: 100%
:alt: VMware Workstation 14 Player - Power off virtual machine
Figure 12: VMware Workstation 14 Player - Power off virtual machine
#. Go to :guilabel:`Power` and select :guilabel:`Shut Down Guest`.
Enable UEFI boot support
************************
|CL| needs UEFI support to boot. To enable UEFI, add the
following line to the end of your VM's :file:`.vmx` file:
.. code-block:: bash
firmware = "efi"
.. note::
Depending on the OS, you can typically find the VMware VM files under:
* On Linux distros: :file:`/home/username/vmware`
* On Windows: :file:`C:\\Users\\username\\Documents\\Virtual Machines`
Power on the VM
***************
After configuring the settings above, power on your |CL| virtual machine.
#. On the :guilabel:`VMware Workstation Player` home screen, select your
VM. See Figure 13.
.. figure:: figures/vmw-player/vmw-player-13.png
:scale: 100%
:alt: VMware Workstation 14 Player - Power on virtual machine
Figure 13: VMware Workstation 14 Player - Power on virtual machine
#. Click :guilabel:`Play virtual machine`.
For other guides on using the VMWare Player and ESXi, see:
* :ref:`vmw-player-preconf`
* :ref:`vmware-esxi-install-cl`
* :ref:`vmware-esxi-preconfigured-cl-image`
.. _VMware ESXi: https://www.vmware.com/products/esxi-and-esx.html
.. _VMware Workstation 14 Player:
https://www.vmware.com/products/workstation-player.html
.. _VMware Workstation Player guide:
https://docs.vmware.com/en/VMware-Workstation-Player/index.html
.. _latest: https://download.clearlinux.org/image/
.. _image: https://download.clearlinux.org/image
@@ -1,116 +0,0 @@
.. _vmware-player:
Use VMware\* Player
###################
This section explains how to run Clear Linux OS for Intel® Architecture
within a `VMware Player`_ environment.
Please ensure you have enabled `Intel® Virtualization Technology
<http://www.intel.com/content/www/us/en/virtualization/virtualization-technology/intel-virtualization-technology.html>`_
(Intel® VT) and `Intel® Virtualization Technology for Directed I/O
<https://software.intel.com/en-us/articles/intel-virtualization-technology-for-directed-io-vt-d-enhancing-intel-platforms-for-efficient-virtualization-of-io-devices>`_
(Intel® VT-d) in your BIOS/UEFI firmware configuration.
Install VMware Player
=====================
VMware Workstation Player, formerly VMware Player, is a virtualization
software package for x64 computers running Microsoft Windows or Linux.
Download VMware player from the `VMware website`_.
Player on Linux
---------------
For the Linux option, you should have a :file:`VMware-Player-{version}_FILE.bundle`
file. To install it, run:
::
$ sudo bash ./VMware-Player-{version}.x86_64.bundle
Player on Windows
-----------------
Follow the instructions from the Setup Assistant.
Prepare Image
=============
#. Download the `latest`_ |CL| **live** version (clear-XXXX-live.img.xz)
#. Decompress the downloaded image. Uncompressed image size is ~ **5GB**.
+ On Linux ::
$ xz -d clear-XXXX-live.img.xz
+ On Windows you can use `7zip`_.
- Right-click the file to *extract in the same directory*.
.. image:: ./figures/7zipwin.png
:alt: 7zip extract here command
#. Convert the installer to :abbr:`VMDK (Virtual Machine Disk)` format.
* On Linux, you can use ``qemu-img convert``::
$ qemu-img convert -O vmdk clear-VERSION-live.img clear.vmdk
* On Windows, you can convert the live image to VMDK format
(from RAW format to VMDK) with a tool like *VBoxManage* from
`VirtualBox`_. You can refer on
:ref:`how to create a VM on VirtualBox <create_vm_vbox>` as example.
Run using VMware Player
=======================
Create a new virtual machine by following the next steps.
#. Launch **VMWare Workstation Player**.
#. On main window click on “Create a new Virtual Machine”.
* Select “**I will install the operating system later**”, and click on
“Next”.
* Select “**Linux**” as “Guest operating system” and version **Other Linux
3.x kernel 64-bit**.
* Type a name for the new virtual machine.
* Perform the *remaining steps* using the default options.
#. Change boot type to EFI. You must change the VMware virtual machine
*configuration* to **Support EFI firmware**; you can do this by editing
the configuration ``.vmx`` file located in the virtual machine folder and
adding the following line::
firmware = "efi"
#. Attach the prepared image as SATA disk. And when you have a new virtual
machine, edit its configuration as follows:
* Click on “Edit virtual machine settings”.
* Remove any default attached hard disk.
* Click on “Add” option below devices list tab and choose Hard disk.
* Choose **SATA** as the virtual disk type.
* Use the existing Clear Linux OS for Intel Architecture virtual disk
The live disk must be set as ``SATA 0:1 Hard Disk (SATA)``; you can
verify this under the “Advanced" section of the disk settings.
Start the virtual machine
=========================
After configuring the settings above, start the virtual machine.
.. _VMware website: https://www.vmware.com/products/player/playerpro-evaluation.html
.. _VMware Player: http://www.vmware.com/products/player/
.. _latest: https://download.clearlinux.org/image/
.. _7zip: http://www.7-zip.org/
.. _VirtualBox: https://www.virtualbox.org/
@@ -33,11 +33,12 @@ SHA512 checksum file, which is designated with the suffix `-SHA512SUMS`.
.. code-block:: console
$ sha512sum ./clear-[version number]-[image type].[compression type] | diff ./clear-[version number]-[image type].[compression type]-SHA512SUMS -
$ sha512sum -c ./clear-[version number]-[image type].[compression type]-SHA512SUMS
If the checksum of the downloaded image is different than the original
checksum, the differences will displayed. Otherwise, an empty output indicates
a match and your downloaded image is good.
checksum, a warning will be displayed with a message indicating the computed
checksum does **not** match. Otherwise, the name of the image will be printed on
the screen followed by `OK`.
Uncompress the Clear Linux image
********************************
@@ -97,6 +97,10 @@ To be able to execute all applications with root privileges, we must add the
Install a GUI to test sudo
--------------------------
.. note::
If you are following this sequence after just setting up the pre-configured VMware virtual machine from
the repo, you must :ref:`increase virtual disk size<increase-virtual-disk-size>` or the following step
will fail.
To test the :command:`sudo` command and ensure it is set up correctly,
install the Gnome Desktop Manager (gdm) and start it.
@@ -13,6 +13,7 @@ maintaining |CLOSIA| after :ref:`installation <get-started>` is completed.
update
bulk-provision
mixer
swupdaddpkg
validate-signatures
telemetry-enable
time
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,148 @@
.. _swupdaddpkg:
Create and add custom bundles to your upstream Clear Linux system
#################################################################
|CLOSIA| offers many curated bundles that you can install on your system to
create your desired capabilities. If the available upstream bundles do not
meet your needs, you can create and add your own custom bundles to your
system using one of two methods. Note: Upstream refers to the official
version of |CL|.
The first method is to use the :ref:`mixer tool<mixer>` to create your own
|CL| image and add your bundles to it. Mixing your own |CL| image can
give you great control and flexibility; however, you must act as an
:abbr:`OSV (Operating System Vendor)` and maintain your releases and
updates because you have forked from upstream.
The second method is to use the :command:`swupd-add-pkg` tool, which also
makes use of mixer to create custom bundles that you can add to your
upstream |CL| system. This simpler method provides a “light” forking from
upstream, which means you can continue to get upstream bundles and updates.
If needed, you can easily revert your system back to the upstream version.
This guide shows you how to accomplish the second method by following these
steps:
#. Set up the workspace.
#. Copy your custom RPM package to the workspace.
#. Create a bundle with your custom RPM package.
#. Migrate your |CL| system to your custom mix.
#. Add your custom bundle to your system.
#. Optional: Revert your system back to 100% upstream.
Set up the workspace
********************
#. Install the mixer bundle to enable mixer.
.. code-block:: console
$ sudo swupd bundle-add mixer
#. Create the workspace.
.. code-block:: console
$ sudo mkdir -p /usr/share/mix/rpms
Copy your custom RPM package to the workspace
*********************************************
.. note::
You cannot simply use RPMs from other Linux distros on |CL|. You must
build RPMs specifically for |CL| in order for them to work properly.
Follow the instructions on how to build RPMs found at the
`Developer tooling framework for Clear Linux`_.
Copy your RPM package to the workspace.
.. code-block:: console
$ sudo cp [RPM] /usr/share/mix/rpms
Create a bundle with your custom RPM package
********************************************
Use the :command:`swupd-add-pkg` command to create a bundle with the RPM
package.
.. code-block:: console
$ sudo swupd-add-pkg [RPM] [bundle-name]
To add more than one RPM to your previously-created bundle, repeat
the :command:`swupd-add-pkg` command and change the RPM name.
.. note::
* If you add the same RPM package more than once, it will simply build a
new mix each time without appending it again to the bundle definition.
* The first time you run the :command:`swupd-add-pkg` command, mixer
creates a new OS version by taking your current upstream |CL| version
and multiplying it by 1000. For example, if your upstream version is
21530, your custom version will be 21530000. For each subsequent call
to swupd-add-pkg, mixer will increment the version by 10. For example,
21530010, 21530020, etc.
Migrate your Clear Linux system to your custom mix
**************************************************
Before you can use your custom bundle, you must migrate your |CL| system
to your custom mix to make the bundle accessible.
.. code-block:: console
$ sudo swupd update --migrate
After you migrate, the version of your |CL| system switches over to your
last custom version number as noted in the previous section.
You can continue to create new bundles with :command:`swupd-add-pkg`
while you are in your custom version of |CL|. You do not need to migrate
again. However, you must run :command:`swupd update` again to update your
system in order to make those bundles visible.
Add your custom bundle to your system
*************************************
#. Get a listing of your newly-created bundle.
.. code-block:: console
$ sudo swupd bundle-list -a
The listing includes all upstream bundles.
#. Add your bundle.
.. code-block:: console
$ sudo swupd bundle-add [bundle-name]
.. note::
You can also update your system to the latest upstream version using
this command:
.. code-block:: console
$ sudo swupd update
Optional: Revert your system back to 100% upstream
**************************************************
If you want to revert your |CL| system back to the official upstream
version, use this command:
.. code-block:: console
$ sudo swupd verify --fix --force --picky -m [upstream-version-number] -C /usr/share/clear/update-ca/Swupd_Root.pem
After the command completes, all custom RPMs and bundles are unavailable
because :file:`/usr/share/mix` is deleted as part of the reversion process.
.. _Developer tooling framework for Clear Linux:
https://github.com/clearlinux/common
@@ -3,41 +3,40 @@
Build a custom Clear Linux based Docker container image
#######################################################
The official base |CLOSIA| container image is published on Docker\* Hub and
is updated on a regular basis. This section contains the steps to build a
custom image.
The official base |CLOSIA| container image is published on Docker\* Hub and is
updated on a regular basis. This guide contains the steps to build a custom
container image.
Prerequisites
*************
* These steps must be performed on a |CL| system because the `swupd` command
is needed to manage bundles in the container.
* The `containers-basic` bundle must be installed on the |CL| system for
Docker to work.
* Basic knowledge of Docker is required.
* You must perform these steps on a |CL| system because the
:abbr:`swupd (software updater)` is used to manage bundles in the
container.
* You must install the :file:`containers-basic` bundle on the |CL| system
or Docker will not work.
* You have a basic understanding of Docker.
Build the base Clear Linux container image
******************************************
Build the base container image
******************************
#. Log in and get root privileges.
.. code-block:: console
.. code-block:: bash
$ sudo -s
sudo -s
#. Verify Docker is installed and running.
.. code-block:: console
.. code-block:: bash
# docker info
docker info
If Docker is installed and running, the expected output will be similar to
this:
If Docker is installed and running, the output is similar to
this example:
.. code-block:: console
# docker info
Containers: 0
Running: 0
Paused: 0
@@ -76,135 +75,135 @@ Build the base Clear Linux container image
127.0.0.0/8
Live Restore Enabled: false
If Docker is not installed, perform these steps:
If Docker is not installed, enter the commands:
.. code-block:: console
.. code-block:: bash
# swupd bundle-add containers-basic
# systemctl start docker
swupd bundle-add containers-basic
systemctl start docker
#. Create the directory structure needed for building the |CL| container.
#. Create the directory structure to build the |CL| container.
.. code-block:: console
.. code-block:: bash
# mkdir -p ./custom-clear-linux-container/base/usr/share/clear/bundles
# cd custom-clear-linux-container
mkdir -p ./custom-clear-linux-container/base/usr/share/clear/bundles
cd custom-clear-linux-container
.. note::
* The directories `customer-clear-linux-container` and `base` are for
the purpose of staging and can be named something else, if preferred.
* The remaining directories (`/usr/share/clear/bundles`) are mandatory.
* The directories :file:`custom-clear-linux-container` and
:file:`base` are used for staging. You can rename these directories.
#. Create the reference files of the minimum required |CL| bundles (`os-core` and
`os-core-update`). `swupd` determines which bundles to download and install
by using the reference filenames.
* The directories :file:`/usr/share/clear/bundles` are mandatory and
cannot be renamed.
.. code-block:: console
#. Create the reference files of the minimum required |CL| bundles,
:file:`os-core` and :file:`os-core-update`. The software updater
uses the reference filenames to determine which bundles to download and
install.
# touch ./base/usr/share/clear/bundles/os-core
# touch ./base/usr/share/clear/bundles/os-core-update
.. note::
.. code-block:: bash
* `os-core` provides the minimal Linux namespace.
* `os-core-update` provides basic suite for running the |CL|
for iA Updater
touch ./base/usr/share/clear/bundles/os-core
touch ./base/usr/share/clear/bundles/os-core-update
#. Optionally, additional bundles can be included with the base image.
.. note::
#. Identify the desired bundles by going to the |CL| website's
:ref:`available-bundles` page or by executing the
`swupd bundle-list -a` command
* :file:`os-core` provides the minimal Linux namespace.
* :file:`os-core-update` provides the basic suite for running the |CLOSIA|
updater.
#. Create the reference files for the identified bundles. For example,
to include the `editors` and `network-basic` bundles:
#. Optionally, you can include additional bundles with the base image.
.. code-block:: console
#. Identify the desired bundles on the |CL| website's
:ref:`available-bundles` page or execute the
:command:`swupd bundle-list -a` command.
# touch ./base/usr/share/clear/bundles/editors
# touch ./base/usr/share/clear/bundles/network-basic
#. Create reference files for the identified bundles. For example,
to include the :file:`editors` and :file:`network-basic` bundles,
enter the commands:
#. Use `swupd` to download and install the bundles into the directory
structure created.
.. code-block:: bash
.. code-block:: console
touch ./base/usr/share/clear/bundles/editors
touch ./base/usr/share/clear/bundles/network-basic
# swupd verify --install --path="base" --manifest 17870 \
#. Use `swupd` to download and install the bundles.
.. code-block:: bash
swupd verify --install --path="base" --manifest 17870 \
--url https://cdn.download.clearlinux.org/update \
--statedir "$PWD/swupd-state" --no-boot-update
.. note::
* `verify -install` tells `swupd` to download and install
* `-path` specifies the root path of where the bundles are to be
installed
* `--manifest` specifies the version of the |CL| bundles to use
* `--url` specifies the URL of the bundles repository
* `--statedir` specifies the state directory where downloaded bundles
and any
state information are stored
* `--no-boot-update` tells `swupd` to skip updating boot files since
it's not needed for a container
The `swupd` example uses the following flags:
For more information on the `swupd` flags, enter the `swupd verify -h`
* :command:`verify -install` tells `swupd` to download and install.
* :command:`--path` specifies the root path where the bundles are to be
installed.
* :command:`--manifest` specifies the version of the |CL| bundles.
* :command:`--url` specifies the URL of the bundles repository.
* :command:`--statedir` specifies the state directory where downloaded bundles
and any state information are stored.
* :command:`--no-boot-update` tells `swupd` to skip updating boot files because
boot files are not required for a container.
For more information on `swupd` flags, enter the :command:`swupd verify -h`
command.
Example output:
.. code-block:: console
.. code-block:: console
# swupd verify --install --path="base" --manifest 17870 \
--url https://cdn.download.clearlinux.org/update \
--statedir "$PWD/swupd-state" --no-boot-update
swupd-client software verify 3.12.2
Copyright (C) 2012-2017 Intel Corporation
swupd-client software verify 3.12.2
Copyright (C) 2012-2017 Intel Corporation
Verifying version 17870
Attempting to download version string to memory
Downloading packs...
Verifying version 17870
Attempting to download version string to memory
Downloading packs...
Extracting python-basic pack for version 17820
...14%
Extracting perl-basic pack for version 17790
...28%
Extracting openssh-server pack for version 17660
...42%
Extracting editors pack for version 17850
...57%
Extracting network-basic pack for version 17650
...71%
Extracting os-core pack for version 17870
...85%
Extracting os-core-update pack for version 17870
...100%
Adding any missing files
...88%
Inspected 33982 files
33974 files were missing
33974 of 33974 missing files were replaced
0 of 33974 missing files were not replaced
Calling post-update helper scripts.
WARNING: boot files update skipped due to --no-boot-update argument
Fix successful
Extracting python-basic pack for version 17820
...14%
Extracting perl-basic pack for version 17790
...28%
Extracting openssh-server pack for version 17660
...42%
Extracting editors pack for version 17850
...57%
Extracting network-basic pack for version 17650
...71%
Extracting os-core pack for version 17870
...85%
Extracting os-core-update pack for version 17870
...100%
Adding any missing files
...88%
Inspected 33982 files
33974 files were missing
33974 of 33974 missing files were replaced
0 of 33974 missing files were not replaced
Calling post-update helper scripts.
WARNING: boot files update skipped due to --no-boot-update argument
Fix successful
.. note::
The `WARNING` message is expected and can be ignored.
#. Tar up the files and compress it.
#. Create a tarball and compress it.
.. code-block:: console
.. code-block:: bash
# tar -C base -cf base.tar .
# xz -v -T0 base.tar
tar -C base -cf base.tar .
xz -v -T0 base.tar
#. Create the Dockerfile to build the image.
.. code-block:: console
.. code-block:: bash
# cat > Dockerfile << EOF
cat > Dockerfile << EOF
FROM scratch
MAINTAINER First Last <first.last@example.com>
ADD base.tar.xz /
@@ -213,169 +212,167 @@ Build the base Clear Linux container image
#. Build the |CL| container image.
.. code-block:: console
.. code-block:: bash
# docker build -t my-custom-clear-linux-container .
docker build -t my-custom-clear-linux-container .
Example output:
.. code-block:: console
.. code-block:: console
# docker build -t my-custom-clear-linux-container .
Sending build context to Docker daemon 806.5MB
Step 1/4 : FROM scratch
--->
Step 2/4 : MAINTAINER First Last <first.last@example.com>
---> Running in 7238f35abcd0
---> ec5064287c60
Removing intermediate container 7238f35abcd0
Step 3/4 : ADD base.tar.xz /
---> 2723b7d20716
Removing intermediate container 16e3ed0df8da
Step 4/4 : CMD /bin/bash
---> Running in efa893350647
---> 5414c3a12993
Removing intermediate container efa893350647
Successfully built 5414c3a12993
Successfully tagged my-custom-clear-linux-container:latest
Sending build context to Docker daemon 806.5MB
Step 1/4 : FROM scratch
--->
Step 2/4 : MAINTAINER First Last <first.last@example.com>
---> Running in 7238f35abcd0
---> ec5064287c60
Removing intermediate container 7238f35abcd0
Step 3/4 : ADD base.tar.xz /
---> 2723b7d20716
Removing intermediate container 16e3ed0df8da
Step 4/4 : CMD /bin/bash
---> Running in efa893350647
---> 5414c3a12993
Removing intermediate container efa893350647
Successfully built 5414c3a12993
Successfully tagged my-custom-clear-linux-container:latest
#. List the newly created |CL| container image.
.. code-block:: console
.. code-block:: bash
# docker images
docker images
Example output:
.. code-block:: console
.. code-block:: console
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
my-custom-clear-linux-container latest 5414c3a12993 About a minute ago 616MB
REPOSITORY TAG IMAGE ID CREATED SIZE
my-custom-clear-linux-container latest 5414c3a12993 About a minute ago 616MB
#. Launch the built |CL| container.
.. code-block:: console
.. code-block:: bash
# docker run -it my-custom-clear-linux-container
docker run -it my-custom-clear-linux-container
Manage bundles in a Clear Linux based container
***********************************************
Manage bundles in a container
*****************************
Bundles can be added and removed from an existing |CL| container by using
the `swupd` command in the Dockerfile.
You can add and remove bundles from a |CL| container using the
:command:`RUN swupd` command in the Dockerfile.
Add a bundle (`swupd bundle-add`)
---------------------------------
Add a bundle
============
This example Dockerfile shows how to add the `pxe-server` bundle to the
previously created |CL| Docker image:
This example Dockerfile adds the :file:`pxe-server` bundle to an existing |CL|
Docker image:
.. code-block:: console
.. code-block:: bash
# cat > Dockerfile << EOF
FROM my-customer-clear-linux-container
MAINTAINER First Last <first.last@example.com>
RUN swupd bundle-add pxe-server
CMD ["/bin/bash/bash"]
EOF
cat > Dockerfile << EOF
FROM my-customer-clear-linux-container
MAINTAINER First Last <first.last@example.com>
RUN swupd bundle-add pxe-server
CMD ["/bin/bash/bash"]
EOF
Example output:
Example output:
.. code-block:: console
.. code-block:: console
# docker build -t my-clearlinux-with-pxe-server-bundle .
docker build -t my-clearlinux-with-pxe-server-bundle .
Sending build context to Docker daemon 806.5MB
Step 1/4 : FROM my-custom-clear-linux-container
---> 5414c3a12993
Step 2/4 : MAINTAINER First Last <first.last@example.com>
---> Running in 19b4411cf4bd
---> 08d400baffde
Removing intermediate container 19b4411cf4bd
Step 3/4 : RUN swupd bundle-add pxe-server
---> Running in 3e634d6e0792
swupd-client bundle adder 3.12.2
Copyright (C) 2012-2017 Intel Corporation
Sending build context to Docker daemon 806.5MB
Step 1/4 : FROM my-custom-clear-linux-container
---> 5414c3a12993
Step 2/4 : MAINTAINER First Last <first.last@example.com>
---> Running in 19b4411cf4bd
---> 08d400baffde
Removing intermediate container 19b4411cf4bd
Step 3/4 : RUN swupd bundle-add pxe-server
---> Running in 3e634d6e0792
swupd-client bundle adder 3.12.2
Copyright (C) 2012-2017 Intel Corporation
Attempting to download version string to memory
Downloading packs...
Attempting to download version string to memory
Downloading packs...
Extracting pxe-server pack for version 17820
.
Installing bundle(s) files...
..............................................................................
..............................................................................
..............................................................................
..............................................................................
..............................................................................
..............................................................................
Calling post-update helper scripts.
WARNING: systemctl not operable, unable to run systemd update triggers
Bundle(s) installation done.
---> 8ead5f2c0c33
Removing intermediate container 3e634d6e0792
Step 4/4 : CMD /bin/bash
---> Running in 0ceae320279b
---> dcd9adb40611
Removing intermediate container 0ceae320279b
Successfully built dcd9adb40611
Successfully tagged my-clearlinux-with-pxe-server-bundle:latest
Extracting pxe-server pack for version 17820
.
Installing bundle(s) files...
..............................................................................
..............................................................................
..............................................................................
..............................................................................
..............................................................................
..............................................................................
Calling post-update helper scripts.
WARNING: systemctl not operable, unable to run systemd update triggers
Bundle(s) installation done.
---> 8ead5f2c0c33
Removing intermediate container 3e634d6e0792
Step 4/4 : CMD /bin/bash
---> Running in 0ceae320279b
---> dcd9adb40611
Removing intermediate container 0ceae320279b
Successfully built dcd9adb40611
Successfully tagged my-clearlinux-with-pxe-server-bundle:latest
.. note::
.. note::
This `WARNING` message is expected and can be ignored because Systemd
doesn't run inside a container.
The `WARNING` message can be ignored because systemd does not run inside
a container.
Remove a bundle (`swupd bundle-remove`)
---------------------------------------
This example Dockerfile shows how to remove the `pxe-server` bundle from the
previously created |CL| Docker image:
Remove a bundle
===============
.. code-block:: console
This example Dockerfile removes the :file:`pxe-server` bundle from an existing
|CL| Docker image:
# cat > Dockerfile << EOF
FROM my-clearlinux-with-pxe-server-bundle
MAINTAINER First Last <first.last@example.com>
RUN swupd bundle-remove pxe-server
CMD ["/bin/bash/bash"]
EOF
.. code-block:: bash
Example output:
cat > Dockerfile << EOF
FROM my-clearlinux-with-pxe-server-bundle
MAINTAINER First Last <first.last@example.com>
RUN swupd bundle-remove pxe-server
CMD ["/bin/bash/bash"]
EOF
.. code-block:: console
Example output:
# docker build -t my-clearlinux-remove-pxe-server-bundle .
.. code-block:: console
Sending build context to Docker daemon 806.5MB
Step 1/4 : FROM my-clearlinux-with-pxe-server-bundle
---> dcd9adb40611
Step 2/4 : MAINTAINER First Last <first.last@example.com>
---> Running in 71b60f15003e
---> 742192751c1a
Removing intermediate container 71b60f15003e
Step 3/4 : RUN swupd bundle-remove pxe-server
---> Running in ad28a3390ecc
swupd-client bundle remover 3.12.2
Copyright (C) 2012-2017 Intel Corporation
docker build -t my-clearlinux-remove-pxe-server-bundle .
Removing bundle: pxe-server
Deleting bundle files...
Total deleted files: 92
Untracking bundle from system...
Success: Bundle removed
1 bundle(s) were removed successfully
---> d6ee7903e14d
Removing intermediate container ad28a3390ecc
Step 4/4 : CMD /bin/bash
---> Running in 7694989e97de
---> ec23189ef954
Removing intermediate container 7694989e97de
Successfully built ec23189ef954
Successfully tagged my-clearlinux-remove-pxe-server-bundle:latest
Sending build context to Docker daemon 806.5MB
Step 1/4 : FROM my-clearlinux-with-pxe-server-bundle
---> dcd9adb40611
Step 2/4 : MAINTAINER First Last <first.last@example.com>
---> Running in 71b60f15003e
---> 742192751c1a
Removing intermediate container 71b60f15003e
Step 3/4 : RUN swupd bundle-remove pxe-server
---> Running in ad28a3390ecc
swupd-client bundle remover 3.12.2
Copyright (C) 2012-2017 Intel Corporation
Also see:
Removing bundle: pxe-server
Deleting bundle files...
Total deleted files: 92
Untracking bundle from system...
Success: Bundle removed
1 bundle(s) were removed successfully
---> d6ee7903e14d
Removing intermediate container ad28a3390ecc
Step 4/4 : CMD /bin/bash
---> Running in 7694989e97de
---> ec23189ef954
Removing intermediate container 7694989e97de
Successfully built ec23189ef954
Successfully tagged my-clearlinux-remove-pxe-server-bundle:latest
* :ref:`cc-getting-started`
For more details, refer to:
* :ref:`cc-getting-started`
* :ref:`architecture-overview`
+165 -155
View File
@@ -1,337 +1,347 @@
.. _dpdk:
Send packages between platforms
###############################
Use DPDK to send packets between platforms
##########################################
:abbr:`Data Plane Development Kit (DPDK)` is a set of libraries and drivers
for fast packet processing. This document describes how to run a basic use
case for **l3fwd DPDK example**. The objective is to *send packages between
two platforms* using a traffic generator called :ref:`pktgen <sec_pktgen>`,
where the l3fwd example application will forward those packages, see
:ref:`figure 1 <f1>`.
This document describes how to send packets between two platforms in the
simple configuration shown in :ref:`Figure 1 <f1>`. The example uses the
:abbr:`Data Plane Development Kit (DPDK)`, which is a set of libraries,
drivers, sample applications, and tools for fast packet processing.
.. _f1:
.. figure:: ./figures/pktgen_lw3fd.png
:align: center
:alt: platform A and B
:alt: Platform A and B
Figure 1: Environment for l3fwd DPDK application.
Figure 1: Environment for l3fwd DPDK application
This example uses the following DPDK components:
**Requirements:**
* pktgen: Traffic generator. See `pktgen documentation`_ for details.
* l3fwd: Layer 3 forwarding example application. See
`l3fwd documentation`_ for details.
* Two platforms using Clear Linux* for Intel® Architecture (recommended
release `13330`_ or higher).
* Both images have the **kernel-native bundle** added.
* Install of **network-basic-dev** bundle:
Prerequisites
*************
.. code-block:: bash
* Two platforms using |CLOSIA| release `13330`_ or higher.
* Both images must include the :file:`kernel-native bundle`.
* Install the :file:`network-basic-dev` bundle with the command:
# swupd bundle-add network-basic-dev
.. code-block:: bash
sudo swupd bundle-add network-basic-dev
* Each platform must have at least one :abbr:`NIC (Network Interface Card)`.
Check the `DPDK project`_ for the list of supported `dpdk.org NICs`_.
* The platforms must have two NICs, at least one each. It's very important to
check network card compatibility with the `DPDK project`_. You can do this
on the `dpdk.org NICS`_ site.
* Two network cables.
Installing dpdk and build l3fwd example (Platform B)
====================================================
Install dpdk and build l3fwd example (Platform B)
*************************************************
#. Move to ``l3fwd`` example.
#. Change to the :file:`l3fwd` example directory.
.. code-block:: bash
# cd /usr/share/dpdk/examples/l3fwd
sudo cd /usr/share/dpdk/examples/l3fwd
#. Assign ``RTE_SDK var`` to the makefiles path.
#. Assign :envvar:`RTE_SDK` variable to the makefiles path.
.. code-block:: bash
# export RTE_SDK=/usr/share/dpdk/
sudo export RTE_SDK=/usr/share/dpdk/
#. Assign ``RTE_TARGET var`` the value where the gcc config file is located.
#. Assign :envvar:`RTE_TARGET` variable to the location of the gcc\* config
file.
.. code-block:: bash
# export RTE_TARGET=x86_64-native-linuxapp-gcc
sudo export RTE_TARGET=x86_64-native-linuxapp-gcc
#. Build the ``l3fwd`` application, then add the configuration header to
the ``CFLAGS`` var.
#. Build the `l3fwd` application and add the configuration header to
the :makevar:`CFLAGS` variable.
.. code-block:: bash
# make CFLAGS+="-include /usr/include/rte_config.h"
sudo make CFLAGS+="-include /usr/include/rte_config.h"
Build pktgen (Platform A)
*************************
.. _sec_pktgen:
Building Pktgen (Platform A)
============================
Since the **pktgen** project is currently not included in Clear Linux OS for
Intel Architecture, you must download it from upstream and build it:
#. Download the `pktgen tar package`_ 3.1.2 or newer.
#. Download the `pktgen tar package`_ v3.1.2 or newer.
#. Decompress packages and move to uncompressed source directory.
#. Assign ``RTE_SDK var`` the path where makefiles are located.
#. Assign :envvar:`RTE_SDK` variable to the path where makefiles are located.
.. code-block:: bash
# export RTE_SDK=/usr/share/dpdk/
sudo export RTE_SDK=/usr/share/dpdk/
#. Assign ``RTE_TARGET var`` the value where the gcc config file is located.
#. Assign :envvar:`RTE_TARGET` to the location of the gcc config file.
.. code-block:: bash
# export RTE_TARGET=x86_64-native-linuxapp-gcc
sudo export RTE_TARGET=x86_64-native-linuxapp-gcc
#. Build pktgen project, and set the ``CONFIG_RTE_BUILD_SHARED_LIB`` variable
with "n".
#. Build the `pktgen` project and set the :makevar:`CONFIG_RTE_BUILD_SHARED_LIB` variable
to "n".
.. code-block:: bash
# make CONFIG_RTE_BUILD_SHARED_LIB=n
sudo make CONFIG_RTE_BUILD_SHARED_LIB=n
Binding NICs to DPDK kernel drivers (Platforms A and B)
=======================================================
Bind NICs to DPDK kernel drivers (Platforms A and B)
****************************************************
The ``l3fwd`` application uses two NICs. DPDK has useful tools for binding
The `l3fwd` application uses two NICs. The DPDK includes tools for binding
NICs to DPDK modules to run DPDK applications.
#. Load the dpdk I/O kernel module
#. Load the DPDK I/O kernel module.
.. code-block:: bash
# modprobe vfio-pci
sudo modprobe vfio-pci
#. Check the status of your NICs; this will show which network cards are not
busy. When another application is using them, the status shows ``Active``,
#. Check the NIC status to determine which network cards are not
busy. When another application is using them, the status shows `Active`,
and those NICs cannot be bound.
.. code-block:: bash
# dpdk-devbind --status
sudo dpdk-devbind --status
#. Bind two available NICs. The general syntax for binding is
**dpdk-devbind --bind=vfio-pci <device-entry>**,
and the following is a working example:
#. Bind two available NICs. The general syntax for binding is:
:command:`dpdk-devbind --bind=vfio-pci <device-entry>`.
A working example is shown below:
.. code-block:: bash
# dpdk-devbind --bind=vfio-pci 01:00.0
sudo dpdk-devbind --bind=vfio-pci 01:00.0
#. Check that your NICs binded correctly by checking the status; ``drv`` should
have ``igb_uio`` value; at this point, the NICs are using the DPDK modules.
#. Check the NIC status to verify that the NICs are bound correctly. If
successful, `drv` displays the value `igb_uio`, which confirms
that the NICs are using the DPDK modules.
Setting hugepages (platforms A and B)
=====================================
Set hugepages (Platforms A and B)
*********************************
Clear Linux OS for Intel Architecture supports ``hugepages`` for the large
memory pool allocation used for packet buffers.
|CLOSIA| supports `hugepages` for the large memory pool allocation used for
packet buffers.
#. Set number of hugepages.
#. Set the number of hugepages.
.. code-block:: bash
# echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
sudo echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
#. Allocate pages on NUMA machines.
.. code-block:: bash
# echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
# echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
sudo echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
sudo echo 1024 > /sys/devices/system/node/node1/hugepages/hugepages-2048kB/nr_hugepages
#. Make memory available for DPDK.
.. code-block:: bash
# mkdir -p /mnt/huge $ mount -t hugetlbfs nodev /mnt/huge
sudo mkdir -p /mnt/huge $ mount -t hugetlbfs nodev /mnt/huge
If you would like to know more about this, refer to the `DPDK guide`_.
For more information, refer to the `DPDK guide`_ System Requirements
section.
Setting a physical environment (Platforms A and B)
==================================================
Set up the physical environment (Platforms A and B)
***************************************************
To achieve the model proposed in the introduction of this topic, (:ref:`f1`),
we need to connect the first Grantleys NICs to the second Grantleys NICs
using the network cables, see :ref:`figure 2<f2>`.
Connect the NICs on Platform A to the NICs on Platform B using the network
cables as shown in :ref:`Figure 2<f2>`.
.. _f2:
.. figure:: ./figures/pyshical_net.png
Figure 2: Physical network environment.
Figure 2: Physical network environment
Running l3fwd application (Platform B)
======================================
Run l3fwd application (Platform B)
**********************************
The ``l3fwd`` application is one of the DPDK examples available when you
install the ``dpdk-dev`` bundle; this application forwards packages from one
NIC to another.
The `l3fwd` application is one of the DPDK examples available when you
install the :file:`dpdk-dev` bundle. `l3fwd` forwards packets from one
NIC to another. For details, refer to the `l3fwd documentation`_.
#. Open the l3fwd example directory.
.. code-block:: bash
# cd /usr/share/dpdk/examples/l3fwd
sudo cd /usr/share/dpdk/examples/l3fwd
#. **This step is very important.** DPDK needs poll drivers for work; these
poll drivers are shared objects in :file:`/usr/lib64`. DPDK supports some
NICs. The full list available at the `dpdk.org NICS`_ docs. You should know
which kernel module the NIC is using and choose a poll driver according to
your NICs.
#. **This step is very important.**
#. At this point the system must have ``hugepages`` requirements. The NICs
bound and the configuration for running ``pktgen`` depends upon network use
cases and available system resources. Use the ``-d`` flag for setting the
pull driver. For example, if the NICs are using ``e1000`` network driver,
they are going to use ``e1000`` poll driver (``librte_pmd_e1000.so``); it
should be in :file:`/usr/lib64` in Clear Linux OS for Intel Architecture,
and it should be enough to add the name. For example
#. DPDK needs poll mode drivers to operate.
#. Poll mode drivers are shared objects in :file:`/usr/lib64`.
#. See the full list of supported NICs at `dpdk.org NICs`_.
#. You must know which kernel module each NIC is using and choose a poll
mode driver that corresponds to your NICs.
#. NIC binding and `pktgen` configuration depends upon network use cases and
available system resources. Use the :command:`-d` flag to set the poll mode
driver.
The following example assumes that the NICs use the `e1000` network driver
and the `e1000` poll mode driver. The :file:`librte_pmd_e1000.so` is
located in :file:`/usr/lib64` in |CL|.
.. code-block:: bash
# ./build/l3fwd -c 0x3 -n 2 -d librte_pmd_e1000.so -- -p 0x3 --config="(0,0,0),(1,0,1)"
sudo ./build/l3fwd -c 0x3 -n 2 -d librte_pmd_e1000.so -- -p 0x3 --config="(0,0,0),(1,0,1)"
#. When the application starts to run, it will show information about the
``l3fwd`` running, so pay attention when the application is Initializing
ports. After port 0 initialization, you'll see a MAC address and the same
for port 1. Save this information for setting configuration to `Pktgen`
project.
#. The `l3fwd` application shows port initialization details at startup.
After port 0 initialization completes, `l3fwd` shows a MAC address and
information for port 1.
Running Pktgen application (Platform A)
===========================================
Save the MAC address for configuring the `pktgen` project.
The `Pktgen` is network traffic generator. It measures the network packaging
performance in a forwarding use case.
Run pktgen application (Platform A)
***********************************
#. At this point the system must have ``hugepages`` requirements and the NICs
bound. The configuration for running ``pktgen`` depends upon the network use
case and the available system resources. The following is a basic
`pktgen` is a network traffic generator included in the DPDK.
#. `pktgen` configuration depends upon the network setup and the
available system resources. The following example shows a basic
configuration.
.. code-block:: bash
# ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xf -n 4 -- -p 0xf -P -m "1.0, 2.1"
sudo ./app/app/x86_64-native-linuxapp-gcc/pktgen -c 0xf -n 4 -- -p 0xf -P -m "1.0, 2.1"
#. Enable active colorful output (optional).
.. code-block:: console
.. code-block:: bash
Pktgen> theme enable
#. The ``l3fwd`` application showed a MAC address per-port initialized; this
MAC addresses should have been set in the pktgen environment::
#. Use the MAC addresses shown by the `l3fwd` application during initialization.
The command to set the MAC addresses in `pktgen` has the format:
> set mac <port number> <mac address>
.. code-block:: bash
And a working example:
set mac <port number> <mac address>
.. code-block:: console
Here is a working example:
.. code-block:: bash
Pktgen> set mac 0 00:1E:67:CB:E8:C9
Pktgen> set mac 1 00:1E:67:CB:E8:C9
#. Start to send packages using the next command:
#. Send packets.
.. code-block:: console
.. code-block:: bash
Pktgen> start 0-1
#. If you followed these steps correctly, you'll see that ``pktgen`` is sending
and receiving packages. For more information, see the `Pktgen
documentation`_.
For more details, see the `pktgen documentation`_.
Annex A: Using pass-through for running on virtual machines
===========================================================
Appendix A: Use pass-through for virtual machines
*************************************************
This section explains how to set up a virtual environment where virtual
machines control the host's NICs.
machines control the NICs on the host.
#. Create a new directory and move to it.
#. Download or create a ``start_qemu.sh`` script for running a kvm virtual
#. Download or create a :file:`start_qemu.sh` script for running a kvm virtual
machine:
.. code-block:: bash
$ curl -O https://download.clearlinux.org/image/start_qemu.sh
sudo curl -O https://download.clearlinux.org/image/start_qemu.sh
#. Download a bare-metal image of Clear Linux OS for Intel Architecture and
rename it as ``clear.img``.
#. Download a bare-metal image of |CLOSIA| and rename it as :file:`clear.img`.
#. Look for an entry for device and vendor & device ID:
#. Look for an Ethernet\* device entry that contains vendor and device ID:
.. code-block:: bash
$ lspci -nn | grep Ethernet
sudo lspci -nn | grep Ethernet
An output example from the last step::
An example output:
.. code-block:: console
03:00.0 Ethernet controller [0200]: Intel Corporation I350 Gigabit Network Connection [8086:1521]
where ``8086:1521`` is ``vendor:device ID`` and ``03:00.0`` is the entry for
device. Make note of this information; it is necessary for unbinding a
host's NICs.
where `03:00.0` is the device entry and `8086:1521` is the `vendor:device
ID`. Record this information, because you need it to unbind the NICs from a
host.
#. Unbind NICs from host to do passthrough with virtual machines. Clear Linux
OS for Intel Architecture currently supports this action. You can use the
following commands::
#. Unbind the NICs from the host to do pass-through with virtual machines. |CLOSIA|
supports this action. The commands take the format:
.. code-block:: bash
echo "vendor device_ID" > /sys/bus/pci/drivers/pci-stub/new_id
echo "entry for device" > /sys/bus/pci/drivers/igb/unbind
echo "entry for device" > /sys/bus/pci/drivers/pci-stub/bind
echo "vendor device_ID" > /sys/bus/pci/drivers/pci-stub/remove_id
Here is a working example:
.. code-block:: bash
$ echo "8086 1521" > /sys/bus/pci/drivers/pci-stub/new_id
$ echo "0000:03:00.0" > /sys/bus/pci/drivers/igb/unbind
$ echo "0000:03:00.0" > /sys/bus/pci/drivers/pci-stub/bind
$ echo "8086 1521" > /sys/bus/pci/drivers/pci-stub/remove_id
sudo echo "8086 1521" > /sys/bus/pci/drivers/pci-stub/new_id
sudo echo "0000:03:00.0" > /sys/bus/pci/drivers/igb/unbind
sudo echo "0000:03:00.0" > /sys/bus/pci/drivers/pci-stub/bind
sudo echo "8086 1521" > /sys/bus/pci/drivers/pci-stub/remove_id
#. Assign to the KVM virtual machine (guest) the unbound NICs previously noted.
Modify the ``start_qemu.sh`` script in ``qemu-system-x86_64`` arguments, and
add the lines with the host's NICs information::
#. Assign the unbound NICs to the KVM virtual machine (guest).
Modify the :file:`start_qemu.sh` script in `qemu-system-x86_64` arguments, and
add the lines with the host's NICs information in the format:
-device pci-assign,host="<entry for device>",id=passnic0,addr=03.0
-device pci-assign,host="<entry for device>",id=passnic1,addr=04.0
.. code-block:: bash
A working example:
-device pci-assign,host="<entry for device>",id=passnic0,addr=03.0
-device pci-assign,host="<entry for device>",id=passnic1,addr=04.0
Here is a working example:
.. code-block:: bash
-device pci-assign,host=03:00.0,id=passnic0,addr=03.0 \
-device pci-assign,host=03:00.3,id=passnic1,addr=04.0 \
#. If you would like to add more NUMA machines to the virtual machine, you can
add the next line in the Makefile boot target::
#. Add more NUMA machines to the virtual machine by adding lines to the
Makefile boot target in the format:
.. code-block:: bash
-numa node,mem=<memory>,cpus=<number of cpus>
As a working example for a virtual machine with 4096 of memory and four CPUs, the configuration
would look like this::
Here is a working example for a virtual machine with 4096 memory and four
CPUs:
.. code-block:: bash
-numa node,mem=2048,cpus=0-1 \
-numa node,mem=2048,cpus=2-3 \
This means that each NUMA machine has to use the same quantity of memory.
.. note:: Each NUMA machine must use the same quantity of memory.
#. Finally, run the ``start_qemu.sh`` script.
#. Run the :file:`start_qemu.sh` script.
.. _13330: https://download.clearlinux.org/releases/13330/
.. _DPDK project: http://dpdk.org
.. _dpdk.org NICS: http://dpdk.org/doc/nics
.. _dpdk.org NICs: http://dpdk.org/doc/nics
.. _pktgen tar package: http://dpdk.org/browse/apps/pktgen-dpdk/refs
.. _DPDK guide: http://dpdk.org/doc/guides/linux_gsg/sys_reqs.html
.. _Pktgen documentation: `Pktgen documentation`_ https://media.readthedocs.org/pdf/pktgen/latest/pktgen.pdf
.. _l3fwd documentation: http://dpdk.org/doc/guides/sample_app_ug/l3_forward.html
.. _pktgen documentation: http://pktgen-dpdk.readthedocs.io/en/latest/index.html
Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 33 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 50 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

+132 -132
View File
@@ -1,70 +1,73 @@
.. _ipxe-install:
Install Clear Linux over the network with iPXE
##############################################
################################################
This guide shows how to install |CL| through :abbr:`PXE (Pre-boot Execution Environment)`.
This guide describes how to install Clear Linux\* using :abbr:`PXE (Pre-boot
Execution Environment)`.
PXE is an industry standard describing the client-server interaction with network-boot software using
the DHCP and TFTP protocols. This guide shows one possible use of this
environment to automatically install |CL|.
PXE is an industry standard that describes client-server interaction with
network-boot software and uses the DHCP and TFTP protocols. This guide shows one
method of using the PXE environment to install |CL|.
The PXE extension known as `iPXE`_\* adds support for additional protocols
such as HTTP, :abbr:`iSCSI (Internet Small Computer Systems Interface)`, :abbr:`AoE (ATA over Ethernet)`, and
:abbr:`FCoE (Fiber Channel over Ethernet)`. iPXE can also be used to enable
network booting on computers which lack built-in PXE support.
The PXE extension called `iPXE`_ adds support for additional protocols such as
HTTP, :abbr:`iSCSI (Internet Small Computer Systems Interface)`, :abbr:`AoE
(ATA over Ethernet\*)`, and :abbr:`FCoE (Fiber Channel over Ethernet\*)`. iPXE
enables network booting on computers with no built-in PXE support.
Figure 1 depicts the flow of information between a PXE server and a PXE
client we must create to install |CL| through iPXE.
To install |CL| through iPXE, you must create a PXE client. Figure 1 depicts
the flow of information between a PXE server and a PXE client.
.. figure:: ./figures/network-boot-flow.png
:alt: PXE information flow
Figure 1: PXE information flow
Figure 1: PXE information flow.
.. caution::
The |CL| image that boots through the PXE process automatically erases all data and partitions on the PXE client system and
creates 3 new partitions to install onto.
The |CL| image that boots through the PXE process automatically erases all
data and partitions on the PXE client system and creates 3 new partitions
to install onto.
Prerequisites
*************
Before booting with iPXE, the following preparations must be made:
Before booting with iPXE, make the following preparations.
* Your PXE server has an Ethernet/LAN boot option.
* Your PXE server has at least two network adapters.
* Your PXE server is connected to a public network.
* Your PXE server and PXE clients are connected to a switch on a private
network.
* Your PXE server has the secure boot option disabled.
* Your PXE clients have a boot order where the network boot option is
prioritized before the disk boot option.
Connect the PXE server and PXE clients to a switch on a private network, as
shown in Figure 2.
.. figure:: ./figures/network-boot-setup.png
:alt: Network topology
Figure 2: Network topology.
Your PXE client must have a boot order where the network boot option is
prioritized before the disk boot option.
Your PXE server must have:
* Ethernet/LAN boot option.
* At least two network adapters.
* Connection to a public network.
* Secure boot option disabled.
.. note::
The ``Secure Boot`` option in the BIOS must be disabled because the UEFI binaries used to
boot |CL| are not signed.
You must disable the secure boot option in the BIOS because the UEFI
binaries used to boot |CL| are not signed.
The required computer and network setup is shown in figure 2.
.. figure:: ./figures/network-boot-setup.png
:alt: NAT network topology
Figure 2: NAT network topology
Configuration
*************
The configuration process to install |CL| using iPXE has been automated with
the :file:`configure-ipxe.sh` script included with
:abbr:`ICIS (Ister Cloud Init Service)`, thus quickly enabling a bulk
provisioning setup. For additional instructions on how to get started with the
script, refer to the guide on the `ICIS GitHub repository`_. Otherwise, to
setup manually, follow the steps below.
To set up |CL| using iPXE automatically, use the :file:`configure-ipxe.sh`
script included with :abbr:`ICIS (Ister Cloud Init Service)`. For additional
instructions on the script, refer to the guide on the `ICIS GitHub repository`_.
#. Define the variables used to parameterize the configuration of an iPXE
boot.
To set up |CL| manually, perform the steps below.
#. Define the variables used for iPXE boot configuration.
.. code-block:: console
@@ -82,28 +85,28 @@ setup manually, follow the steps below.
#. Log in and get root privilege.
.. code-block:: console
.. code-block:: bash
$ sudo -s
sudo -s
#. Add the ``pxe-server`` bundle to your |CL| system. This bundle has all the
#. Add the `pxe-server` bundle to your |CL| system. The bundle contains all
files needed to run a PXE server.
.. code-block:: console
.. code-block:: bash
# swupd bundle-add pxe-server
sudo swupd bundle-add pxe-server
#. Download the latest network-bootable release of |CL| and extract the
files.
.. code-block:: console
.. code-block:: bash
# mkdir -p $ipxe_root
# curl -o /tmp/clear-pxe.tar.xz \
sudo mkdir -p $ipxe_root
sudo curl -o /tmp/clear-pxe.tar.xz \
https://download.clearlinux.org/current/clear-$(curl \
https://download.clearlinux.org/latest)-pxe.tar.xz
# tar -xJf /tmp/clear-pxe.tar.xz -C $ipxe_root
# ln -sf $(ls $ipxe_root | grep 'org.clearlinux.*') $ipxe_root/linux
sudo tar -xJf /tmp/clear-pxe.tar.xz -C $ipxe_root
sudo ln -sf $(ls $ipxe_root | grep 'org.clearlinux.*') $ipxe_root/linux
.. note::
@@ -118,8 +121,8 @@ setup manually, follow the steps below.
.. code-block:: console
# cat > $ipxe_root/ipxe_boot_script.ipxe << EOF
#!ipxe
sudo cat > $ipxe_root/ipxe_boot_script.ipxe << EOF
sudo!ipxe
kernel linux quiet init=/usr/lib/systemd/systemd-bootchart \
initcall_debug tsc=reliable no_timer_check noreplace-smp rw \
initrd=initrd
@@ -127,14 +130,14 @@ setup manually, follow the steps below.
boot
EOF
#. The ``pxe-server`` bundle contains a lightweight web-server known as
``nginx``. Create a configuration file for ``nginx`` to serve |CL| to PXE
#. The `pxe-server` bundle contains a lightweight web-server known as
`nginx`. Create a configuration file for `nginx` to serve |CL| to PXE
clients with the following contents:
.. code-block:: console
# mkdir -p /etc/nginx/conf.d
# cat > /etc/nginx/conf.d/$ipxe_app_name.conf << EOF
sudo mkdir -p /etc/nginx/conf.d
sudo cat > /etc/nginx/conf.d/$ipxe_app_name.conf << EOF
server {
listen $ipxe_port;
server_name localhost;
@@ -145,42 +148,42 @@ setup manually, follow the steps below.
}
EOF
# cp /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf
sudo cp /usr/share/nginx/conf/nginx.conf.example /etc/nginx/nginx.conf
.. note::
Creating a separate configuration file for ``nginx`` to serve
network-bootable images on a non-standard port number preserves
existing `nginx` configurations.
Create a separate `nginx` configuration file to serve network-bootable
images on a non-standard port number. This action saves existing `nginx`
configurations.
#. Start ``nginx`` and enable the startup on boot option.
#. Start `nginx` and enable the startup on boot option.
.. code-block:: bash
sudo systemctl start nginx
sudo systemctl enable nginx
#. The `pxe-server` bundle contains a lightweight DNS server which
conflicts with the DNS stub listener provided in `systemd-resolved`.
Disable the DNS stub listener and temporarily stop `systemd-resolved`.
.. code-block:: console
# systemctl start nginx
# systemctl enable nginx
#. The ``pxe-server`` bundle contains a lightweight DNS server which
conflicts with the DNS stub listener provided by ``systemd-resolved``.
Disable the DNS stub listener and temporarily stop ``systemd-resolved``.
.. code-block:: console
# mkdir -p /etc/systemd
# cat > /etc/systemd/resolved.conf << EOF
sudo mkdir -p /etc/systemd
sudo cat > /etc/systemd/resolved.conf << EOF
[Resolve]
DNSStubListener=no
EOF
# systemctl stop systemd-resolved
sudo systemctl stop systemd-resolved
#. Assign a static IP address to the network adapter for the private network
and restart ``systemd-networkd`` with the following commands:
and restart `systemd-networkd` with the following commands:
.. code-block:: console
# mkdir -p /etc/systemd/network
# cat > /etc/systemd/network/70-internal-static.network << EOF
sudo mkdir -p /etc/systemd/network
sudo cat > /etc/systemd/network/70-internal-static.network << EOF
[Match]
Name=$internal_iface
[Network]
@@ -188,74 +191,73 @@ setup manually, follow the steps below.
Address=$pxe_internal_ip/$pxe_subnet_bitmask
EOF
# systemctl restart systemd-networkd
sudo systemctl restart systemd-networkd
#. Configure NAT to route traffic from the private network to the public
network, effectively turning the PXE server into a router. To keep these
changes in spite of reboots, save the changes to the firewall with the
following commands:
#. Configure :abbr:`NAT (Network Address Translation)` to route traffic from
the private network to the public network. This action makes the PXE
server act as a router. To make these changes persistent during reboots, save the
changes to the firewall with the following commands:
.. code-block:: console
.. code-block:: bash
# iptables -t nat -F POSTROUTING
# iptables -t nat -A POSTROUTING -o $external_iface -j MASQUERADE
# systemctl enable iptables-save.service
# systemctl restart iptables-save.service
# systemctl enable iptables-restore.service
# systemctl restart iptables-restore.service
sudo iptables -t nat -F POSTROUTING
sudo iptables -t nat -A POSTROUTING -o $external_iface -j MASQUERADE
sudo systemctl enable iptables-save.service
sudo systemctl restart iptables-save.service
sudo systemctl enable iptables-restore.service
sudo systemctl restart iptables-restore.service
.. note::
The firewall masks or translates packets to make them appear as
coming from the PXE server. Thus, it hides the PXE clients from the
public network.
The firewall masks packets to make them appear as coming from the PXE
server and hides PXE clients from the public network.
#. Configure the kernel to forward network packets to different
interfaces. Otherwise, NAT will not work.
.. code-block:: console
.. code-block:: bash
# mkdir -p /etc/sysctl.d
# echo net.ipv4.ip_forward=1 > /etc/sysctl.d/80-nat-forwarding.conf
# echo 1 > /proc/sys/net/ipv4/ip_forward
sudo mkdir -p /etc/sysctl.d
sudo echo net.ipv4.ip_forward=1 > /etc/sysctl.d/80-nat-forwarding.conf
sudo echo 1 > /proc/sys/net/ipv4/ip_forward
#. The ``pxe-server`` bundle contains iPXE firmware images that allow computers
#. The `pxe-server` bundle contains iPXE firmware images that allow computers
without an iPXE implementation to perform an iPXE boot. Create a TFTP
hosting directory and populate it with the iPXE firmware images with the
following commands:
hosting directory and populate the directory with the iPXE firmware images
with the following commands:
.. code-block:: console
.. code-block:: bash
# mkdir -p $tftp_root
# ln -sf /usr/share/ipxe/undionly.kpxe $tftp_root/undionly.kpxe
sudo mkdir -p $tftp_root
sudo ln -sf /usr/share/ipxe/undionly.kpxe $tftp_root/undionly.kpxe
#. The ``pxe-server`` bundle contains a lightweight TFTP, DNS, and DHCP
server known as ``dnsmasq``. Create a configuration file for ``dnsmasq``
#. The `pxe-server` bundle contains a lightweight TFTP, DNS, and DHCP
server known as `dnsmasq`. Create a configuration file for `dnsmasq`
to listen on a dedicated IP address for those functions. PXE clients on
the private network will use this IP address to access those functions.
the private network will use this IP address.
.. code-block:: console
# cat > /etc/dnsmasq.conf << EOF
sudo cat > /etc/dnsmasq.conf << EOF
listen-address=$pxe_internal_ip
EOF
#. Add the options to serve iPXE firmware images to PXE clients over TFTP to
the ``dnsmasq`` configuration file.
the `dnsmasq` configuration file.
.. code-block:: console
# cat >> /etc/dnsmasq.conf << EOF
sudo cat >> /etc/dnsmasq.conf << EOF
enable-tftp
tftp-root=$tftp_root
EOF
#. Add the options to host a DHCP server for PXE clients to the ``dnsmasq``
#. Add the options to host a DHCP server for PXE clients to the `dnsmasq`
configuration file.
.. code-block:: console
# cat >> /etc/dnsmasq.conf << EOF
sudo cat >> /etc/dnsmasq.conf << EOF
dhcp-leasefile=/var/db/dnsmasq.leases
dhcp-authoritative
@@ -272,7 +274,7 @@ setup manually, follow the steps below.
EOF
This configuration provides the following important functions:
The configuration provides the following important functions:
* Directs PXE clients without an iPXE implementation to the TFTP server
to acquire architecture-specific iPXE firmware images that allow them
@@ -281,44 +283,42 @@ setup manually, follow the steps below.
defined subnet.
* Directs PXE clients to the DNS server.
* Directs PXE clients to the PXE server for routing via NAT.
* Divides the private network into two pools of IP addresses, one for
network booting and another for usage after boot, each with their own
lease times.
* Divides the private network into two pools of IP addresses. One pool
is for network boot and one pool is used after boot. Each pool has
their own lease times.
#. Create a file where ``dnsmasq`` can record the IP addresses it provides
#. Create a file for `dnsmasq` to record the IP addresses it provides
to PXE clients.
.. code-block:: console
.. code-block:: bash
# mkdir -p /var/db
# touch /var/db/dnsmasq.leases
sudo mkdir -p /var/db
sudo touch /var/db/dnsmasq.leases
#. Start ``dnsmasq`` and enable startup on boot.
#. Start `dnsmasq` and enable startup on boot.
.. code-block:: console
.. code-block:: bash
# systemctl enable dnsmasq
# systemctl restart dnsmasq
sudo systemctl enable dnsmasq
sudo systemctl restart dnsmasq
#. Start ``systemd-resolved``.
#. Start `systemd-resolved`.
.. code-block:: console
.. code-block:: bash
# systemctl start systemd-resolved
sudo systemctl start systemd-resolved
.. note::
Using the ``dnsmasq`` DNS server allows ``systemd-resolved`` to dynamically
update the list of DNS servers for the private network from the public
network. This setup effectively creates a pass-through DNS server which
relies on the DNS servers listed in :file:`/etc/resolv.conf`.
`systemd-resolved` dynamically updates the list of DNS servers for the
private network if you use the `dnsmasq` DNS server. The setup creates a
pass-through DNS server that relies on the DNS servers listed in
:file:`/etc/resolv.conf`.
#. Power on the PXE client and watch it boot and install |CL|.
#. Power on the PXE client and watch the client boot and install |CL|.
.. note::
After booting, |CL| will automatically partition the hard drive,
install itself, update to the latest version, and reboot.
After booting, |CL| automatically partitions the hard drive,
installs itself, updates to the latest version, and reboots.
**Congratulations!** You have successfully installed and configured a PXE
@@ -1,31 +1,48 @@
.. _network-bonding:
Combine multiple interfaces
###########################
Combine multiple interfaces with network bonding
################################################
Network bonding is a technique for combining multiple network interfaces into
a single, logical interface, providing some redundancy and bandwidth
aggregation.
Network bonding combines multiple network interfaces into a single logical
interface to provide redundancy and bandwidth aggregation.
|CLOSIA| includes the bonding_ and team_ drivers. The guide example provided
below shows how to configure systemd to use the ``bonding`` driver.
|CLOSIA| includes Linux bonding_ and team_ drivers. This guide describes how
to configure systemd to use the `bonding` driver.
The example demonstrates how to:
* Bond all four ports of a quad-port NIC in `802.3ad` mode.
* Enable jumbo frames to optimize large data transfers on the local network.
Your NICs and network switch must support `802.3ad` mode and jumbo frames. The
example explains how to configure your NICs for both features. Your switch may
require additional configuration. See your switch documentation for details.
.. note::
All commands in this guide must be run as root.
1. Create the ``/etc/systemd/network`` directory (if it doesn't already exist):
You must run all commands in this guide as root.
#. Log in and get root privileges.
.. code-block:: console
# mkdir -p /etc/systemd/network
sudo -s
This directory contains the configuration files and network settings
for the virtual device and its underlying physical interfaces.
#. Create the :file:`/etc/systemd/network` directory.
2. Configure systemd to create a virtual network device, ``bond1``. Use a text
editor to create a file named ``30-bond1.netdev`` as shown here:
.. code-block:: bash
.. code-block:: ini
mkdir -p /etc/systemd/network
The :file:`/etc/systemd/network` directory contains configuration files and
network settings for the virtual device and its underlying physical
interfaces.
#. Configure systemd to create a virtual network device called `bond1`. Use a
text editor to create a file named :file:`30-bond1.netdev`.
.. code-block:: console
[NetDev]
Name=bond1
@@ -37,19 +54,15 @@ below shows how to configure systemd to use the ``bonding`` driver.
MIIMonitorSec=1s
LACPTransmitRate=fast
The syntax for this file is defined in the systemd.netdev_ manpage.
`This example`__ may be used verbatim, or tuned to your particular
requirements. Note that ``802.3ad`` mode requires explicit support from
your NICs and network switch. This and other modes may also require
additional configuration of your network switch.
Refer to the systemd.netdev_ manpage for :file:`30-bond1.netdev` file
syntax. This example is based on Example 9 on the manpage. Modify the
example for your configuration.
__ https://www.freedesktop.org/software/systemd/man/systemd.netdev.html#id-1.20.10
#. Configure the slave interfaces. Create a text file named
:file:`30-bond1-enp1s0.network`. Assign the slave interfaces to the virtual
`bond1` device and use the syntax shown in systemd.network_.
3. Configure the slave interfaces, assigning them to the new ``bond1`` device,
using the syntax in systemd.network_, and in a text file named
``30-bond1-enp1s0.network`` as shown here:
.. code-block:: ini
.. code-block:: console
[Match]
Name=enp1s0f*
@@ -60,21 +73,20 @@ __ https://www.freedesktop.org/software/systemd/man/systemd.netdev.html#id-1.20.
[Link]
MTUBytes=9000
This example demonstrates bonding all four ports of a quad-port NIC, with
names in the range ``enp1s0f0-enp1s0f3``, allowing the use of a single file
with a wildcard match. You may also create a separate file for each NIC,
particularly if they have names that are not wildcard-friendly. This
configuration assigns each NIC as a slave of ``bond1``. For best results,
do not assign addresses or DHCP support to the individual NICs.
The example bonds all four ports of a quad-port NIC as a slave of `bond1`.
The example uses a wildcard match because the NIC names are in the range
`enp1s0f0-enp1s0f3`. If your NIC names are not wildcard-compatible, create
a separate :file:`.network` file for each NIC.
This example also enables jumbo frames of up to 9000 bytes to optimize large
data transfers on the local network. Again, your NICs and switch must
support jumbo frames, and your switch may require additional configuration.
For best results, do not assign addresses or DHCP support to the individual
NICs.
4. Define the network configuration for the bonded interface in a file named
``30-bond1.network`` as shown here:
The `MTUBytes` setting enables jumbo frames of up to 9000 bytes. Your
switch may require additional configuration to support this setting.
.. code-block:: ini
#. Configure the bonded interface in a file named :file:`30-bond1.network`.
.. code-block:: console
[Match]
Name=bond1
@@ -86,25 +98,26 @@ __ https://www.freedesktop.org/software/systemd/man/systemd.netdev.html#id-1.20.
[Link]
MTUBytes=9000
Since ``bond1`` is a virtual interface, it has no concept of physical link
status. The ``BindCarrier`` directive indicates that the link status of this
interface is determined by the status of the listed slave devices.
`bond1` is a virtual interface with no physical link status.
This is the logical interface, so assign it an IP address. DHCP is more
complicated with bonded interfaces, and is not covered in this example.
`BindCarrier` indicates that the `bond1` link status is determined by the
status of the listed slave devices.
This file also enables jumbo frames of up to 9000 bytes. This option must be
enabled for all slave interfaces *and* the bonded interface, in order to take
effect.
`Address` contains an IP address that you assign to the logical interface.
DHCP bonded interfaces are complex and outside the scope of this example.
5. Apply the new network configuration:
`MTUBytes` must be set to 9000 on all slave interfaces and on the bonded
interface for successful jumbo frames operation. If `MTUBytes` is not the
same on all interfaces, then the lowest value is used.
.. code-block:: console
#. Apply the new network configuration with the command:
# systemctl restart systemd-networkd
.. code-block:: bash
The MTU settings will not take effect until a reboot, or if you explicitly
apply them via ``ifconfig``, for example.
systemctl restart systemd-networkd
The `MTUBytes` settings do not take effect until you reboot or manually
apply the settings with a utility such as `ifconfig`.
.. _bonding:
https://www.kernel.org/doc/Documentation/networking/bonding.txt
@@ -7,10 +7,11 @@ This guide provides step-by-step instructions for common tasks associated with
the configuration, administration, and use of networks in the |CLOSIA|.
.. toctree::
:maxdepth: 1
:maxdepth: 1
ipxe-install
dpdk
ovs-dpdk
network-bonding
custom-clear-container
ipxe-install
dpdk
ovs-dpdk
network-bonding
custom-clear-container
vnc
+790
View File
@@ -0,0 +1,790 @@
.. _vnc:
Remote-desktop to a Clear Linux host using VNC
##############################################
:abbr:`VNC (Virtual Network Computing)` is a client-server GUI-based tool
that allows you to connect via remote-desktop to your |CLOSIA| host.
This guide shows you how to:
* Install the VNC server and misc. components on your |CL| host.
* Configure a VNC-server-start method on your |CL| host.
* Install a VNC viewer app and an SSH client on your client system.
* Establish a VNC connection to your |CL| host.
* Terminate a VNC connection to your |CL| host.
* Encrypt VNC traffic through an SSH tunnel.
Install the VNC server and misc. components on your Clear Linux host
********************************************************************
To configure VNC to work on your |CL| host, install these bundles:
* `desktop-autostart`: Installs :abbr:`GDM (Gnome Desktop Manager)`, sets
it to start automatically on boot, and installs TigerVNC Viewer.
* `vnc-server`: Installs the TigerVNC server.
Follow these steps:
#. Log into your |CL| host and get root privileges.
.. code-block:: console
$ sudo -s
#. Install the |CL| bundles.
.. code-block:: console
# swupd bundle-add desktop-autostart vnc-server
#. Reboot your |CL| host.
Configure a VNC-server-start method on your Clear Linux host
************************************************************
There are three methods you can use to configure and start the VNC server on
your host:
.. list-table:: Table 1: VNC-server-start Configuration Methods
:widths: 10,20,20,20
:header-rows: 1
* - Attribute
- `Method 1`: Manually start a VNC session
- `Method 2`: Automatically start a VNC session via a systemd service script
- `Method 3`: Create multi-user logins with authentication through GDM
* - Description
- This is the traditional method where you SSH into the |CL| host, manually
start a VNC session to get a display ID, and connect to it by
supplying the display ID.
- The system administrator sets up a systemd service script for you with
a pre-assigned display ID. You make a VNC connection and supply
your pre-assigned display ID.
- The system adminstrator configures GDM to accept connection requests.
When you make a VNC connection to the |CL| host, you see
the GDM login screen and authenticate as if you are local.
* - Who configures VNC settings?
- You
- System adminstrator
- System adminstrator
* - Who starts VNC session?
- You
- Set to start automatically on boot by system administrator
- Set to start automatically on boot by system administrator
* - Who ends VNC sesssion?
- You
- You
- System administrator can disable VNC service altogether
* - Requires VNC password to authenticate?
- Yes
- Yes
- No. Use |CL| account username and password through GDM
Although all three methods can coexist on the same |CL| host, we recommend
you pick a method that suits your needs.
For simplicity, the rest of this guide refers to these methods as
`Method 1`, `Method 2`, and `Method 3`.
Method 1: Manually start a VNC session
======================================
You (and each user) must perform these steps to initialize your VNC settings.
#. Log in.
#. Open a terminal emulator.
#. Start VNC with the :command:`vncserver` command. Since this is your
first time starting VNC, it adds default configuration files and asks you
to set a VNC password.
.. code-block:: console
$ vncserver
Example output:
.. code-block:: console
$ vncserver
You will require a password to access your desktops.
Password:
Verify:
Would you like to enter a view-only password (y/n)? n
xauth: file /home/vnc-user-a/.Xauthority does not exist
New 'clr-linux:2 (vnc-user-a)' desktop is clr-linux:2
Creating default startup script /home/vnc-user-a/.vnc/xstartup
Creating default config /home/vnc-user-a/.vnc/config
Starting applications specified in /home/vnc-user-a/.vnc/xstartup
Log file is /home/vnc-user-a/.vnc/clr-linux:2.log
Upon completion, you can find the default configuration files and the
password file hidden in the `.vnc` directory in your home directory.
Also, a VNC session starts and shows a unique display ID, which is the
number following the hostname and the colon `:`. In the above example, the display ID is 2. In a later step, you will supply the display ID to
your VNC viewer app for connection.
#. Kill the active VNC session for the time being with the
:command:`vncserver -kill :[display ID]` command. Substitute [display ID]
with your active VNC session display ID. For example:
.. code-block:: console
$ vncserver -kill :2
.. note::
If you do not recall the active session display ID, use the
:command:`vncserver -list` command to find it.
#. Optional configurations:
* To customize settings such as screen size, security type, etc.,
modify the :file:`$HOME/.vnc/config` file.
* To customize the applications to run at startup, modify the
:file:`$HOME/.vnc/xstartup` file.
Method 2: Automatically start a VNC session via a systemd service script
========================================================================
To configure VNC for this method, you must have root privileges. You will
set up a systemd service file for all intended VNC users with their own
preassigned unique display ID.
#. Log in and get root privileges.
.. code-block:: console
$ sudo -s
#. Make sure the user accounts already exist. Use the following command to
list all users.
.. code-block:: console
# cut -d: -f1 /etc/passwd
#. Create the path :file:`/etc/systemd/system`.
.. code-block:: console
# mkdir -p /etc/systemd/system
#. Create a systemd service script file :file:`vncserver@:[X].service`,
where [X] is the display ID, for each user in :file:`/etc/systemd/system`
Each user must be assigned a unique display ID. Be sure the correct
username is entered in the `User` field. The example below shows user
`vnc-user-b` who is assigned the display ID `5`.
.. code-block:: console
# cat > /etc/systemd/system/vncserver@:5.service << EOF
[Unit]
Description=VNC Remote Desktop Service for "vnc-user-b" with display ID "5"
After=syslog.target network.target
[Service]
Type=simple
User=vnc-user-b
PAMName=login
PIDFile=/home/%u/.vnc/%H%i.pid
ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 || :'
ExecStart=/usr/bin/vncserver %i -geometry 2000x1200 -alwaysshared -fg
ExecStop=/usr/bin/vncserver -kill %i
[Install]
WantedBy=multi-user.target
EOF
#. Have each user log into their account and set a VNC password with
the :command:`vncpasswd` command before proceeding to the next step.
#. Start the VNC service script and set it to start automatically on
boot for each user. Replace the [X] with the display ID.
.. code-block:: console
# systemctl daemon-reload
# systemctl start vncserver@:[X].service
# systemctl enable vncserver@:[X].service
#. After starting the services, verify they are running.
.. code-block:: console
# systemctl | grep vnc
The example below shows 2 VNC sessions that were successfully started for
users `vnc-user-b` with display ID 5 and `vnc-user-c` with display ID 6.
.. code-block:: console
# systemctl | grep vnc
vncserver@:5.services loaded active running VNC Remote Desktop Service for "vnc-user-b" with display ID "5"
vncserver@:6.services loaded active running VNC Remote Desktop Service for "vnc-user-c" with display ID "6"
system-vncserver.slice loaded active active system-vncserver.slice
Method 3: Multi-user logins with authentication through GDM
===========================================================
For this method, VNC is configured as a systemd service that listens on port
5900 and GDM is configured to accept access requests from VNC. When you
make a VNC connection to your |CL| host, you are presented with the GDM login screen and you authenticate as if you are local. You must have root privileges to perform this configuration.
#. Log in and get root privileges.
.. code-block:: console
$ sudo -s
#. Create the path :file:`/etc/systemd/system`.
.. code-block:: console
# mkdir -p /etc/systemd/system
#. Create a systemd socket file :file:`xvnc.socket` and add the following:
.. code-block:: console
# cat > /etc/systemd/system/xvnc.socket << EOF
[Unit]
Description=XVNC Server on port 5900
[Socket]
ListenStream=5900
Accept=yes
[Install]
WantedBy=sockets.target
EOF
#. Create a systemd service file :file:`xvnc@.service` and add the following:
.. code-block:: console
# cat > /etc/systemd/system/xvnc@.service << EOF
[Unit]
Description=Daemon for each XVNC connection
[Service]
ExecStart=-/usr/bin/Xvnc -inetd -query localhost -geometry 2000x1200 -once -SecurityTypes=None
User=nobody
StandardInput=socket
StandardError=syslog
EOF
#. Create the path :file:`/etc/gdm`.
.. code-block:: console
# mkdir -p /etc/gdm
#. Create a GDM :file:`custom.conf` file and add the following:
.. code-block:: console
# cat > /etc/gdm/custom.conf << EOF
[xdmcp]
Enable=true
Port=177
EOF
#. Start the VNC socket script and set it to start automatically on boot.
.. code-block:: console
# systemctl daemon-reload
# systemctl start xvnc.socket
# systemctl enable xvnc.socket
#. After starting the socket, verify it is running.
.. code-block:: console
# systemctl | grep vnc
The example below shows the xvnc.socket is running.
.. code-block:: console
# systemctl | grep vnc
xvnc.socket loaded active listening XVNC Server on port 5900
system-xvnc.slice loaded active active system-xvnc.slice
See the `vncserver` Man page for additional information.
Install a VNC viewer app and an SSH client on your client system
****************************************************************
You need a VNC viewer app on your client system to connect to your |CL| host.
An SSH client is only needed if you chose to use `Method 1` or you plan to
encrypt your VNC traffic, which is discussed later in this guide.
Perform the steps below to add these apps to your client system.
Install a VNC viewer app
========================
On |CL|:
.. code-block:: console
# swupd bundle-add desktop-autostart
On Ubuntu, Mint:
.. code-block:: console
# apt-get install xtightvncviewer
On Fedora:
.. code-block:: console
# dnf install tigervnc
On Windows:
* Install `RealVNC for Windows`_
On macOS:
* Install `RealVNC for macOS`_
Install an SSH client
=====================
* On most Linux distros (Clear Linux, Ubuntu, Mint, Fedora, etc.) and macOS,
SSH is built-in so you don't need to install it.
* On Windows, you can install `Putty`_.
Establish a VNC connection to your Clear Linux host
***************************************************
Depending on the VNC-server-configuration method chosen, use the appropriate VNC connection:
If you chose `Method 1`, you must take a few extra steps by
using SSH to connect to your |CL| host and then manually launching VNC.
If you chose `Method 2`, get your preassigned VNC display ID from your
system administrator first and then proceed to the
:ref:`connect-to-vnc-session` section below.
If you chose `Method 3`, proceed to the
:ref:`connect-to-vnc-session` below.
SSH into your Clear Linux host and launch VNC
=============================================
#. SSH into your Clear Linux host
#. On Linux distros and macOS:
.. code-block:: console
$ ssh [username]@[clear-linux-host-ip-address]
#. On Windows:
#. Launch Putty.
#. Under the :guilabel:`Category` section, select :guilabel:`Session`.
See Figure 1.
#. Enter the IP address of your Clear Linux host in the
:guilabel:`Host Name (or IP address)` field.
#. Set the :guilabel:`Connection type` option to :guilabel:`SSH`.
#. Click the :guilabel:`Open` button.
.. figure:: figures/vnc/vnc-1.png
:scale: 90 %
:alt: Putty - configure SSH session settings
Figure 1: Putty - configure SSH session settings
#. Log in with your |CL| username and password. Do not use your VNC
password.
#. Start a VNC session.
.. code-block:: console
$ vncserver
Example output:
.. code-block:: console
$ vncserver
New 'clr-linux:3 (vnc-user-c)' desktop is clr-linux:3
Starting applications specified in /home/vnc-user-c/.vnc/xstartup
Log file is /home/vnc-user-c/.vnc/clr-linux:3.log
#. Take note of the generated display ID because you will input it into
the VNC viewer app to establish the connection. The above example shows
the display ID is 3.
.. note::
VNC automatically picks a unique display ID unless you specify one.
To specify a display ID, enter a unique number that is not already
in use after the colon. For example:
.. code-block:: console
$ vncserver :8
#. You can now end the SSH connection by logging out. This does
not terminate your active VNC session.
.. _connect-to-vnc-session:
Connect to your VNC session
===========================
For `Method 1` and `Method 2`, you must connect to a specific active session
or display ID using one of two options:
* Use a fully-qualified VNC port number, which consists of the default VNC
server port (5900) plus the display ID
* Use the display ID
For example, if the display ID is 3, it can be specified as `5903` or just
as `3`. For `Method 3`, VNC does not expect a display ID. Use `5900`. For simplicity, the instructions below use the fully-qualified VNC port
number.
**On Linux distros:**
#. Open a terminal emulator and enter:
.. code-block:: console
$ vncviewer [clear-linux-host-ip-address]:[fully-qualified VNC port number]
#. Enter your credentials.
* For `Method 1` and `Method 2`, enter your VNC password. No username is
required.
* For `Method 3`, enter your |CL| account username and password through
GDM.
.. note::
With `Method 3`, you cannot remotely log into your |CL| host through
VNC if you are logged in locally and vice versa.
**On Windows and macOS using `RealVNC` app:**
#. Start the RealVNC viewer app. See Figure 2.
#. Enter the IP address of the Clear Linux host and the fully-qualified
VNC port number.
The following screenshot shows connecting to |CL| host
192.168.25.54 with a fully-qualified VNC port number 5902.
.. figure:: figures/vnc/vnc-2.png
:scale: 90 %
:alt: RealVNC Viewer
Figure 2: RealVNC Viewer
#. Press the :kbd:`Enter` key.
#. Enter your credentials.
* For `Method 1` and `Method 2`, enter your VNC password. No username is
required.
* For `Method 3`, enter your |CL| account username and password through
GDM.
.. note::
With `Method 3`, you cannot remotely log into your |CL| host through
VNC if you are logged in locally and vice versa.
`Optional: Configure RealVNC Image Quality`
-------------------------------------------
To increase the RealVNC viewer image quality, manually change the `ColorLevel` value. Follow these steps:
#. Right-click a connection node and select :guilabel:`Properties...`.
See Figure 3.
.. figure:: figures/vnc/vnc-3.png
:scale: 90 %
:alt: RealVNC Viewer - change connection node properties
Figure 3: RealVNC Viewer - change connection node properties
#. Select the :guilabel:`Expert` tab. See Figure 4.
#. Select the :guilabel:`ColorLevel` setting and change it to your
preferred setting.
.. figure:: figures/vnc/vnc-4.png
:scale: 90 %
:alt: RealVNC Viewer - change ColorLevel
Figure 4: RealVNC Viewer - change :guilabel:`ColorLevel`
Terminate a VNC connection to your Clear Linux host
***************************************************
For `Method 1` and `Method 2`, once started, a VNC session remains active
on your |CL| host even if you close your VNC viewer app. If you want to
truly terminate an active VNC session, follow these steps:
#. SSH into your Clear Linux host.
#. Open a terminal emulator.
#. Find the active VNC session display ID with the command
:command:`vncserver -list`.
.. code-block:: console
$ vncserver -list
#. Terminate it with the :command:`vncserver -kill` command followed by a
colon and the display ID.
.. code-block:: console
$ vncserver -kill :[display ID]
#. For `Method 3`, only the system administrator can stop and disable the
VNC service by using these commands:
.. code-block:: console
# systemctl stop xvnc.socket
# systemctl disable xnvc.socket
Encrypt VNC traffic through an SSH tunnel
*****************************************
By default, VNC traffic is not encrypted. Figure 6 shows an example warning
from RealVNC Viewer.
.. figure:: figures/vnc/vnc-6.png
:scale: 90 %
:alt: RealVNC Viewer - Connection not encrypted warning
Figure 6: RealVNC Viewer - Connection not encrypted warning
To add security, VNC traffic can be routed through an SSH tunnel. This is accomplished by following these steps:
#. Configure the VNC server to only accept connection from localhost by
adding the `-localhost` option.
#. Set up an SSH tunnel between your client system and your |CL| host.
Your client system will forward traffic from the localhost (the client)
destined for a specified fully-qualified VNC port number (on the client)
to your |CL| host with the same port number.
#. The VNC viewer app on your client system will now connect to localhost,
instead of the IP address of your |CL| host.
Configure VNC to only accept connection from localhost
======================================================
For `Method 1`:
#. Edit the :file:`config` file located in :file:`$HOME/.vnc` and uncomment
the `# localhost` line. It should look like this:
.. code-block:: console
## Supported server options to pass to vncserver upon invocation can be listed
## in this file. See the following manpages for more: vncserver(1)
Xvnc(1).
## Several common ones are shown below. Uncomment and modify to your liking.
##
# securitytypes=vncauth,tlsvnc
# desktop=sandbox
# geometry=2000x1200
localhost
# alwaysshared
#. If an active session exists, kill it, and then restart it.
For `Method 2`:
#. Edit the systemd service script :file:`vncserver@:[X].service` located in
:file:`/etc/systemd/system` and add `-localhost` to the `ExecStart`
line. The example below uses vncserver@:5.service:
.. code-block:: console
[Unit]
Description=VNC Remote Desktop Service for "vnc-user-b" with display ID "5"
After=syslog.target network.target
[Service]
Type=simple
User=vnc-user-b
PAMName=login
PIDFile=/home/%u/.vnc/%H%i.pid
ExecStartPre=/bin/sh -c '/usr/bin/vncserver -kill %i > /dev/null 2>&1 || :'
ExecStart=/usr/bin/vncserver %i -geometry 2000x1200 -localhost -alwaysshared -fg
ExecStop=/usr/bin/vncserver -kill %i
[Install]
WantedBy=multi-user.target
#. Restart the service script:
.. code-block:: console
# systemctl daemon-load
# systemctl restart vncserver@:5.service
For `Method 3`:
#. No change is needed to the :file:`xvnc@service` script.
After you have restarted your VNC session, you can verify that it only
accepts connections from localhost by using the :command:`netstat`
command like this:
.. code-block:: console
$ netstat -plant
.. note::
Add the |CL| `network-basic` bundle to get the :command:`netstat`
command.
Figure 7 shows two VNC sessions (5901 and 5905) accepting connections from
any host as specified by the `0.0.0.0`'s. This is before the `-localhost` option was used.
.. figure:: figures/vnc/vnc-7.png
:scale: 100 %
:alt: VNC session accepting connection from any host
Figure 7: VNC sessions (5901 and 5905) accepting connections from any host
Figure 8 shows two VNC sessions (5901 and 5905) only accepting connections from localhost as specified by `127.0.0.1`'s. This is after the `-localhost` option was used.
.. figure:: figures/vnc/vnc-8.png
:scale: 100 %
:alt: VNC session only accepting connection from localhost
Figure 8: VNC sessions (5901 and 5905) only accepting connections from localhost
Set up an SSH tunnel from your client system to your |CL| host
==============================================================
**On Linux distros and macOS:**
#. Open terminal emulator and enter:
.. code-block:: console
$ ssh -L [client port number]:localhost:[fully-qualified VNC port number] \
-N -f -l [username] [clear-linux-host-ip-address]
#. Enter your |CL| account password (not your VNC password).
.. note::
* `-L` specifies that [client port number] on the localhost (on the
client side) is forwarded to [fully-qualified VNC port number]
(on the server side).
* Replace `[client port number]` with an available client port number
(for example: 1234). For simplicity, you can make the
`[client port number]` the same as the `[fully-qualified VNC port number]`.
* Replace `[fully-qualified VNC port number]` with 5900 (default VNC
port) plus the display ID. For example, if the display ID is 2,
the fully-qualified VNC port number is is 5902.
* `-N` tells SSH to only forward ports and not execute a remote
command.
* `-f` tells SSH to go into the background before command execution.
* `-l` specifies the username to log in as.
**On Windows:**
#. Launch Putty.
#. Specify the |CL| VNC host to connect to.
#. Under the :guilabel:`Category` section, select :guilabel:`Session`.
See Figure 1.
#. Enter the IP address of your Clear Linux host in the
:guilabel:`Host Name (or IP address)` field.
#. Set the :guilabel:`Connection type` option to :guilabel:`SSH`.
#. Configure the SSH tunnel. See Figure 9 for an example.
#. Under the :guilabel:`Category` section, go to
:guilabel:`Connection` > :guilabel:`SSH` > :guilabel:`Tunnels`.
#. In the :guilabel:`Source port` field, enter an available client
port number (for example: 1234). For simplicity, you can make the
`Source port` the same as the fully-qualified VNC port number.
#. In the :guilabel:`Destination` field, enter
`localhost:` plus the fully-qualified VNC port number.
#. Click the :guilabel:`Add` button.
.. figure:: figures/vnc/vnc-9.png
:scale: 100 %
:alt: Putty - configure SSH tunnel
Figure 9: Putty - configure SSH tunnel
#. Click the :guilabel:`Open` button.
#. Enter your |CL| account password (not your VNC password).
Connect to a VNC session through an SSH tunnel
==============================================
After you have set up an SSH tunnel, follow these instructions to connect to
your VNC session.
**On Linux distros:**
#. Open terminal emulator and enter:
.. code-block:: console
$ vncviewer localhost:[client port number]
**On Windows and macOS using `RealVNC`:**
#. Start the RealVNC viewer app.
#. Enter `localhost` and the fully-qualified VNC port number. See Figure 10
for an example.
.. figure:: figures/vnc/vnc-10.png
:scale: 100 %
:alt: RealVNC viewer app connecting to localhost:1234
Figure 10: RealVNC viewer app connecting to `localhost:1234`
.. note::
RealVNC will still warn that the connection is not encrypted even
though its traffic is going through the SSH tunnel. You can ignore
this warning.
.. _RealVNC for Windows: https://www.realvnc.com/en/connect/download/viewer/windows/
.. _RealVNC for macOS: https://www.realvnc.com/en/connect/download/viewer/macos/
.. _Putty: https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
@@ -45,8 +45,8 @@ Follow these guidelines when creating graphics for the |CLOSIA|:
* Use only approved image formats. Use either PNG or JPEG bitmap files for
screenshots and SVG files for vector graphics. If a figure is not a
photograph or screenshot, please provide figure as a vector graphic to
ensure it can be changed later on.
photograph or screenshot, use the vector graphic file format to ensure
the figure can be changed later.
Examples
@@ -3,10 +3,11 @@
Language reference guide
########################
This section provides you with the accepted use of the English language. It
includes information about words use, punctuation, and grammar. This guide
does not replace a professional writer's review but is intended to help
collaborators submit consistent contributions.
This section describes acceptable usage of the English language in the
|CLOSIA| documentation. It includes information about words use,
punctuation, and grammar. This guide does not replace a professional
writer's review but is intended to help collaborators submit consistent
contributions.
To make translations easier and to make the content accessible to non-native
speakers, |CLOSIA| uses Simple English. However, we have not implemented any
@@ -13,8 +13,8 @@ makes translation easier, and improves comprehension for people whose
first language is not English.
|CLOSIA| does not use controlled language, which restricts the writer's
vocabulary to a list of approved words. Some preferences are evidently in
place.
vocabulary to a list of approved words. However, we do strongly recommend
using the language principles described below.
Short sentences and paragraphs
******************************
@@ -26,7 +26,7 @@ principle of one main idea in a sentence, plus one additional point if
needed.
Similarly, restrict your paragraph length to about six sentences.
Remember the basic structure of a paragraph: Introduction, body and
Remember the basic structure of a paragraph: Introduction, body, and
conclusion. Both the introduction and the conclusion should be one
sentence long. The body of a paragraph should never exceed four
sentences. Here less is more.
@@ -17,8 +17,8 @@ Follow these general guidelines:
* Indent the contents correctly. This allows the content to be read even if
it is not rendered.
* Only create a table if the body of the table is larger than 6, that means
at least 2x3 or 3x2.
* Only create a table if the body of the table contains six or more cells,
which is a minimum table size of at least 2x3 or 3x2.
ReST supports several types of tables. |CL| uses grid and
:abbr:`CSV-tables (Comma Separated Values tables)`. Grid tables are only
@@ -71,9 +71,9 @@ support several layout options. For example:
crunchy, now would it?"
"Gannet Ripple", 1.99, "On a stick!"
CSV-tables can have a title, the header row is optional and separate from the
rest of the table, each column's width is customizable among others. See the
Sphinx `CSV-tables documentation`_ to learn all the possible options
Some of the options available with CSV-tables are table titles, an optional
header row separate from the rest of the table, and customizable column width.
See the Sphinx `CSV-tables documentation`_ to learn all the possible options
available.
This template can help you create CSV-tables:
@@ -198,13 +198,13 @@ other instances available, they are also listed but not selected.
chmod 400 AWSClearTestKey.pem
#. Copy the text highlighted in the :guilabel:`Example:` section that is
shown in :ref:`figure 11<fig-aws-web-11>`. Paste the copied text into your terminal and add
`-l clear` text to the end of the command string. Press the :kbd:`Enter`
key to execute the command.
shown in :ref:`figure 11<fig-aws-web-11>`. Paste the copied text into your
terminal, change the text before the `@` sign to the username `clear`, and
press the :kbd:`Enter` key to execute the command.
.. code-block:: console
ssh -i "AWSClearTestKey.pem" root@ec2-34-209-39-184.us-west-2.compute.amazonaws.com -l clear
ssh -i "AWSClearTestKey.pem" clear@ec2-34-209-39-184.us-west-2.compute.amazonaws.com
#. A message appears on the terminal stating the authenticty of the host can't
be established and prompts you with the message:
+134
View File
@@ -0,0 +1,134 @@
.. _spark:
Set up a standalone cluster system using Apache\* Spark\*
#########################################################
This tutorial describes how to install, configure, and run Apache Spark on
|CLOSIA|. Apache Spark is a fast general-purpose cluster computing system with
the following features:
* Provides high-level APIs in Java\*, Scala\*, Python\*, and R\*.
* Includes an optimized engine that supports general execution graphs.
* Supports high-level tools including Spark SQL, MLlib, GraphX, and Spark
Streaming.
In this tutorial, you will install Spark on a single machine running the
master daemon and a worker daemon.
Prerequisites
*************
This tutorial assumes you have installed |CL| on your host system.
For detailed instructions on installing |CL| on a bare metal system, visit
the :ref:`bare metal installation tutorial<bare-metal-install>`.
Before you install any new packages, update |CL| with the following command:
.. code-block:: bash
sudo swupd update
Install Apache Spark
********************
Apache Spark is included in the :file:`big-data-basic` bundle. To install the
framework, enter:
.. code-block:: bash
sudo swupd bundle-add big-data-basic
Configure Apache Spark
**********************
#. Create the configuration directory with the command:
.. code-block:: bash
sudo mkdir /etc/spark
#. Copy the default templates from :file:`/usr/share/defaults/spark` to
:file:`/etc/spark` with the command:
.. code-block:: bash
sudo cp /usr/share/defaults/spark/* /etc/spark
.. note:: Since |CL| is a stateless system, you should never modify the
files under the :file:`/usr/share/defaults` directory. The software
updater overwrites those files.
#. Copy the template files below to create custom configuration files:
.. code-block:: bash
sudo cp /etc/spark/spark-defaults.conf.template /etc/spark/spark-defaults.conf
sudo cp /etc/spark/spark-env.sh.template /etc/spark/spark-env.sh
sudo cp /etc/spark/log4j.properties.template /etc/spark/log4j.properties
#. Edit the :file:`/etc/spark/spark-env.sh` file and add the
:envvar:`SPARK_MASTER_HOST` variable. Replace the example address below
with your localhost IP address. View your IP address using the
:command:`hostname -I` command.
.. code-block::
SPARK_MASTER_HOST="10.300.200.100"
.. note:: This optional step enables the master's web user interface to
view information needed later in this tutorial.
#. Edit the :file:`/etc/spark/spark-defaults.conf` file and update the
`spark.master` variable with the `SPARK_MASTER_HOST` address and port `7077`.
.. code-block::
spark.master spark://10.300.200.100:7077
Start the master server and a worker daemon
*******************************************
#. Start the master server using:
.. code-block:: bash
sudo /usr/share/apache-spark/sbin/./start-master.sh
#. Start one worker daemon and connect it to the master using the
`spark.master` variable defined earlier:
.. code-block:: bash
sudo /usr/share/apache-spark/sbin/./start-slave.sh spark://10.300.200.100:7077
#. Open an internet browser and view the worker daemon information using
the master's IP address and port `8080`:
.. code-block::
http://10.300.200.100:8080
Run the Spark wordcount example
*******************************
#. Run the wordcount example using a file on your local host and output the
results to a new file with the following command:
.. code-block:: bash
sudo spark-submit /usr/share/apache-spark/examples/src/main/python/wordcount.py ~/Documents/example_file > ~/Documents/results
#. Open an internet browser and view the application information using
the master's IP address and port `8080`:
.. code-block::
http://10.300.200.100:8080
#. View the results of the wordcount application in the :file:`~/Documents/results` file.
**Congratulations!**
You successfully installed and set up a standalone Apache Spark cluster.
Additionally, you ran a simple wordcount example.
@@ -13,6 +13,7 @@ also redirect where records go if they wish to collect records for themselves
by setting up and using their own telemetry backend server.
A telemetry backend server consists of two Flask applications:
* The :guilabel:`collector` is an ingestion app for records received from the
:guilabel:`telemetrics-client` probes.
* The :guilabel:`telemetryui` web app exposes several visualizations of the
+2 -1
View File
@@ -18,4 +18,5 @@ specific |CLOSIA| use cases.
fmv
aws-web/aws-web
telemetry-backend/telemetry-backend
smb-file-sharing/smb-file-sharing
smb/smb
spark