clearlinux/clear-linux-documentation
2
0
Fork 0
mirror of https://github.com/clearlinux/clear-linux-documentation.git synced 2026-09-06 13:51:40 +00:00
Code Issues Packages Projects Releases Wiki Activity

adding latest version of CC Architecture overview with req consistent use of Intel Clear Containers

Browse Source 
Signed-off-by: Leona <leonax.cook@intel.com>
This commit is contained in:
Leona
2016-08-12 11:55:22 -07:00
parent 15ba5c5641
commit ca20b647af
1 changed files with 407 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
source/clear-containers.rst
+407 -54
View File
@@ -1,85 +1,438 @@
.. _clear-containers:
.. _clear_containers.rst:
Getting started with Intel® Clear Containers
############################################
Intel® Clear Containers
#######################
Intel® Clear Containers for Docker* Engine is now available
for multiple operating systems. This enables executing existing
Docker applications in the secure and fast Intel Clear Containers
environment.
Introduction
============
Binary packages
===============
Intel® Clear Containers is a collection of tools, configurations,
and techniques anchored on an implementation that leverages Intel®
Architecture to optimize container launching and execution workflow. These optimizations improve speed, size, and efficiency while offering a number
of benefits that can be derived only from hardware-backed virtual machines
(hardware-enforced isolation and security, for example) on Intel® VT
technology.
The primary host platform is Clear Linux* Project for Intel®
Architecture, version 4000 or better. However, binaries for a
range of operating systems are available from:
These methods are applied across all levels of the host/virtual machine
hierarchy: from the host-side userland software stack down through the host
Linux* kernel, and into the client-side kernel and userland.
- https://software.opensuse.org/package/clear-containers-docker
Although it is available as a standalone offering, the Clear Containers
technology works best when it is able to leverage optimizations designed
into the Clear Linux Project.
Currently experimental builds are available for:
Customers can integrate all or parts of Intel Clear Containers into a
container infrastructure.
- CentOS*, Scientific Linux* 7
- Fedora* 21, 22, 23
- openSUSE* 13.1, 13.2, Tumbleweed
- SUSE* Linux Enterprise 12
- Debian* 8.0
- Ubuntu* 15.04
.. _architecture_overview.rst:
If you have any feedback, please mail it to the
dev@lists.clearlinux.org mailing list. Subscription to this
list is available at: https://lists.clearlinux.org/mailman/listinfo/dev.
Architecture Overview
=====================
Installation instructions
=========================
Intel Clear Containers are architected around the Linux
:abbr:`Kernel Virtual Machine (KVM)` virtualization infrastructure to
make best use of Intel Architecture VT features. Operational speed
gets improved and overhead gets reduced by optimizing existing code,
removing redundant components, and implementing new techniques for
containers with :abbr:`KVM (Kernel Virtual Machine)`.
Using hosts other than Clear Linux OS for Intel Architecture
------------------------------------------------------------
Version 1.0 of Clear Containers was designed as a lightweight container
system based around `kvmtool`_'s ``lkvm``,
:abbr:`KVM (Kernel Virtual Machine)` and Intel VT-x features; the
initial version was aimed primarily at Docker* integration. Version
2.0 replaces ``lkvm`` with a lightweight version of
:abbr:`QEMU (Quick EMUlator)` `(link) <http:www.qemu.org>`_.
If you are *not* using Clear Linux OS for Intel Architecture, follow the instructions below:
Version 2.0 also expands the feature set to include key technologies, such
as `SR-IOV`_, and the :abbr:`Open Container Initiative (OCI)` runtime API.
#. Visit the link below and select your operating system by clicking the appropriate icon:
https://software.opensuse.org/download.html?project=home%3Aclearlinux%3Apreview&package=clear-containers-docker
#. Follow the brief instructions shown.
#. If your host is behind a proxy, you will need to add the proxies to the docker service. Create ``/etc/systemd/system/docker.service.d/proxy.conf`` and add the next lines to the file::
V1.0
====
[Service]
Environment=HTTP_PROXY=http://proxy.example.com:80/
Environment=NO_PROXY=localhost,127.0.0.1
V1.0 (also known as **Intel® Clear Containers for Docker
Engine**) is based around `kvmtool`_, with example host integrations for
Docker and `rkt`_.
#. Reload your systemd configuration::
.. figure:: _static/images/clearcontainersV1.svg
:align: center
:alt: Intel Clear Containers V1.0
$ sudo systemctl daemon-reload
#. Start the Docker service::
Host kernel optimizations
-------------------------
$ systemctl restart docker
Intel Clear Containers operate better when a number of host kernel features and
optimizations are applied:
Using Clear Linux OS for Intel Architecture as Host
---------------------------------------------------
* Enabling :abbr:`Kernel Samepage Merging (KSM)` in the host kernel
is recommended for efficient page sharing of VM pages. Kernel documentation
can be found in Documentation/vm/ksm.txt Config symbol: ``CONFIG_KSM``
* Using a kernel version >= v4.0 (or backporting appropriate
patches if your kernel version is less than v4.0), to get the best
:abbr:`KVM (Kernel Virtual Machine)` VM startup times
If you are running Clear Linux OS for Intel Architecture on your
host system, follow the instructions below:
.. note::
#. Enable the repository by running the following as the ``root`` user::
Intel :abbr:`Extended Page Table (EPT)` acceleration will be
automatically detected and used by your host kernel if supported
by your hardware. You can check whether this feature is present by
looking for the ``ept`` string in the :file:`/proc/cpuinfo` of your
system. See `mmu.txt`_ for more details.
# swupd bundle-add containers-basic
#. Reload your systemd configuration::
Host user space
---------------
$ sudo systemctl daemon-reload
Intel Clear Containers V1.0 host user space is based around `kvmtool`_ as a fast
and lightweight hypervisor. Optimizations to `kvmtool`_ include:
#. Start the Docker service::
* **File access**, enabling efficient *shmem* / *pci-bar* / :abbr:`Direct
Access (DAX)` file access to client.
* **Less verbosity**.
* **Minimal UART scanning** to improve speed.
* **TSC timer functionality changes** passing the client apic timer
calibration step speeds up container creation time.
* Adding ability to **skip unused features**, (such as creation of a
custom rootfs).
* **Removing need for BIOS** saves boot time.
* **No bootloader required** speeds up initial booting of a machine.
* **Direct kernel boot** -- The hypervisor can boot the kernel directly as
an uncompressed ELF binary. Although the kernel image is slightly larger
than a compressed one, it is faster to read and boot the larger
file than it is to uncompress and boot the slightly smaller file.
$ systemctl restart docker
Source Code
===========
Client mini-OS
--------------
The experimental source code is based on the Docker version 1.9.0
upstream release and is available at:
Intel Clear Containers V1.0 uses an optimized client user space (mini-OS) as its
primary launch vehicle to execute workload commands. The mini-OS is built
with a Clear Linux distribution that has an optimized configuration for
time and space efficiency. The mini-OS includes:
- https://github.com/clearlinux/docker
* Minimized ``systemd`` configuration
* Optimized ``libc``
* Custom AutoFDO settings
* Optimized multi-lib runtime support
* Optimized kernel config (speed and size)
The mini-OS configuration can be modified and rebuilt by customers for their
own use cases, which may preclude the need to load further client images.
Client customer images
----------------------
Intel Clear Containers V1.0 mini-OS workloads can be used to bootstrap further
customer images. These customer images would generally be mapped into the
client via the host filesystem using :abbr:`9p (Plan 9 9p remote filesystem
protocol)`, :abbr:`DAX (Direct Access)` or other filesystem and virtual
device interfaces. These customer images could, for example:
* Mount a new subtree containing a payload and execute it.
* Mount a new subsystem and chroot to it for contained execution.
The mini-OS image has been optimized for size and speed. It may be replaced
or superseded -- in whole or in part -- by customer-created images. Keep
in mind, of course, that any benefits the mini-OS provides may be lost
unless equivalent optimizations exist in the customer-created image, or have
been migrated into the image they create.
V2.0
====
Intel Clear Containers V2.0 adopts an optimized version of the established `QEMU`_
host virtualization engine, in order to support extra features not found in
Clear Containers V1.0. Clear Containers. V2.0 is also compatible with the
:abbr:`OCI (Open Container Initiative)` runtime-specification standard,
introducing a host-side abstraction tool to ease host-side integration and to
isolate integration instances from future changes to the underlying Clear
Containers architecture.
.. figure:: _static/images/clearcontainersV2.svg
:align: center
:alt: Clear Containers V2.0
Host kernel optimizations
-------------------------
V2.0 host kernel optimizations are currently the same as
the V1.0 optimizations.
Host user space
---------------
Host user space is based around an optimized version of `QEMU`_ called
``qemu-lite``, with an :abbr:`OCI (Open Container Initiative)`
runtime-compliant wrapper called ``cor``.
Our version of ``qemu-lite`` has the following modifications:
* :abbr:`DAX (Direct Access)` support, **enabling fast and space efficient**
file access through zero-copy mapping and multi-container sharing of raw
client filesystem images from the host filesystem.
* **Reduced "slimline" PC model** to reduce startup costs in both `QEMU`_
and the client kernel.
* **Removed need for BIOS**, saving boot time.
* **No bootloader requirement**, to speed up boot.
* **Reduced memory footprint** by disabling memory-hungry features that
are not required by the client system.
* **Direct kernel boot**, allowing fast booting by loading the kernel as
an uncompressed ELF binary. Although the kernel image is slightly larger
than a compressed one, it is faster to read and boot the larger
file than it is to uncompress and boot the slightly smaller file.
* **Added an** :abbr:`OCI (Open Container Initiative)` **runtime-compliant
wrapper**, AKA ``cor``, for easier integration with
:abbr:`OCI (Open Container Initiative)`-compliant host orchestration systems.
Client mini-OS
--------------
The Client mini-OS is based on the same Clear Linux OS-based system as
used in Intel Clear Containers V1.0; however, it may be built from more
recent versions and with more current components, such as the kernel version.
Client customer images
----------------------
Client customer images are supported in the same manner as they are
in V1.0.
Architectural component details
===============================
Host kernel components
----------------------
:abbr:`Kernel SamePage Merging (KSM)`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Linux Kernel Documentation: Documentation/vm/ksm.txt
:abbr:`KSM (Kernel Samepage Merging)` allows the kernel to locate
and merge (share) identical memory pages within the system, even
when they are not sourced from the same binary. When sourced from
the same binary, the kernel will naturally share through the
:abbr:`copy-on-write (COW)` method.
:abbr:`KSM (Kernel Samepage Merging)` also allows the kernel to
localize and to coalesce pages from within virtual machine memory
spaces that would not normally be shared, thus saving memory space.
To enable :abbr:`KSM (Kernel Samepage Merging)`, check that your host kernel
config includes ``CONFIG_KSM``, and that your host system is running the
``ksmd`` daemon.
:abbr:`EPT (Extended Page Tables)`
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Linux Kernel Documentation: Documentation/virtual/kvm/mmu.txt
:abbr:`EPT (Extended Page Tables)` is an acceleration technology for virtual
machine memory mappings. It reduces the number of Virtual Machine Manager
entry/exits from the host system, thus improving system performance. If your
hardware system supports :abbr:`EPT (Extended Page Tables)`, you'll see the
``ept`` feature listed in the ``/proc/cpuinfo`` information from your system.
The kernel, :abbr:`KVM (Kernel Virtual Machine)` and `QEMU`_ will
automatically use and benefit from :abbr:`EPT (Extended Page Tables)`
when supported by your system hardware.
You can also check on the `Intel ARK website`_ to see if your Intel CPU
supports **Intel VT-x with Extended Page Tables**; check under the
*Advanced Technologies* table on the specific page for your CPU.
:abbr:`KVM (Kernel Virtual Machine)` startup optimizations
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Host kernel startup was optimized before the Linux kernel v4.0
release by removing some unnecessary ``synchronize_rcu()`` calls. You
should ensure your kernel is at least v4.0, or that you have backported
any appropriate patches to your host kernel: the ``synchronize_rcu() opt``,
at the very least.
.. We should add a Persistent data (how do we do that on R/O or COW'd
filesystems for instance?
[do we have a standard pattern to do for these docs?]
Persistence
~~~~~~~~~~~
Host tooling
------------
Kvmtool
~~~~~~~
Kvmtool is used in Intel Clear Containers V1.0 for virtual machine
configuration and management. It was chosen because it is lighter
and faster than the alternatives, and it's also easy to modify.
Modifications to `kvmtool`_ include:
* Implementation of **copy-free** :abbr:`DAX (Direct Access)` **file-system
access**.
* **Less verbosity**.
* **Minimal UART scanning** to improve speed.
* **TSC timer functionality changes** passing the client apic timer
calibration step speeds up container creation time.
* Adding ability to **skip unused features**, (such as creation of a
custom rootfs).
* **Removing need for BIOS** saves boot time.
* **No bootloader required** speeds up initial booting of a machine.
* **Direct kernel boot** -- The hypervisor can boot the kernel directly as
an uncompressed ELF binary. Although the kernel image is slightly larger
than a compressed one, it ends up being faster to read and boot the larger
file than it is to uncompress and boot the slightly smaller file.
.. _qemu-lite:
qemu-lite
~~~~~~~~~
``qemu-lite`` is a modified version of `QEMU`_ used for the virtual
machine configuration and management in Intel Clear Containers 2.0.
The modifications made beyond generic `QEMU`_ are described in the
following sections:
:abbr:`DAX (Direct Access)` enablement
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
:abbr:`DAX (Direct Access)` enablement under ``qemu-lite`` utilizes
existing `QEMU`_ ``nvdimm memdev`` functionality.
PC-lite
^^^^^^^
A new `QEMU`_ PC model, called ‘pc-lite’, has been added that removes
all unused or unnecessary PC style elements from the machine emulation
that are not required for the client VM. This improves both speed of
execution and memory footprint.
Cor
^^^
Cor (the Clear :abbr:`OCI (Open Container Initiative)` runtime manager)
implements the :abbr:`OCI (Open Container Initiative)` runtime specification
atop of the V2.0 infrastructure (such as ``qemu-lite``). By
utilizing Cor, your :abbr:`OCI (Open Container Initiative)`-compliant system
can be implemented with Clear Containers whilst also insulating
the user against any future underlying changes in Clear Containers,
thus allowing easier future integration of upgrades. Cor currently
supports :abbr:`OCI (Open Container Initiative)` runtime version 0.6.0.
Client components
~~~~~~~~~~~~~~~~~
The client-side components consist of the mini-OS kernel and root
filesystem, and optionally further customer specific items, such as
a further fuller distribution or system to load. The intention is
that customers may either extend and expand the mini-OS as required,
or they can use the mini-OS to further load a complete self-contained
image of their choice.
Client mini-OS
^^^^^^^^^^^^^^
The mini-OS is an optimized version of Clear Linux OS for Intel Architecture
which has been designed for the fastest and smallest container boot. The
mini-OS consists of a Linux kernel image and root filesystem image.
* **Kernel** -- The mini-OS's kernel is a Clear Linux kernel containing
the minimum feature set required to boot the client container. The kernel
has optimized for space and speed. This kernel can be modified and
re-built as desired, for specific requirements.
* **DAX** -- The :abbr:`Direct Access (DAX)` filesystem.
(Linux Kernel Documentation: ``Documentation/filesystems/dax.txt``).
Mapping host-side files into the memory map of the client allows the use of
:abbr:`DAX (Direct Access)` to directly mount those files, bypassing the
client side page cache and the virtual device mechanisms between host and
client. This allows efficient zero-copy mapping and replaces costly virtual
device manipulations with efficient page fault handling, thus being faster
and more space-efficient than other filesystem mount methods. :abbr:`DAX
(Direct Access)` is enabled in Intel Clear Containers V1.0 using a shmem
PCI-BAR mechanism configured by `kvmtool`_.
.. figure:: _static/images/dax-v1.svg
:align: center
:abbr:`DAX (Direct Access)` is enabled in Intel Clear Containers
V2.0 using an NVDIMM `QEMU`_ memdev mechanism:
.. figure:: _static/images/dax-v2.svg
:align: center
:abbr:`DAX (Direct Access)` can only be used to mount single flat files
from the host side (such as uncompressed filesystems), and not trees of
files in the host filesystem. More than one :abbr:`DAX (Direct Access)`
mount can be utilized though. :abbr:`DAX (Direct Access)` is limited only
by the virtual address space available, so it can easily accommodate large
file mappings.
:abbr:`DAX (Direct Access)` support was introduced in v4.0 of the kernel.
Also see the `qemu-lite`_ section.
* **Rootfs image** -- The mini-OS rootfs image is a Clear Linux
rootfs. It can execute the client workload and be modified and
extended using the bundle method to enable further features as
necessary. It can also be used to further execute another client
container image, such as a different Linux distribution.
Customer Client images and workloads
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Customers may utilize their own client images by instructing
the mini-OS to execute them using as the mini-OS workload. Please
refer to the `Intel Clear Containers integration guide`_ for
further detail.
.. removed this section since it is in the GSG
FAQ
===
Q. **"Can I run Clear Containers on any host Linux?"**
A. Yes, any up-to date or recent Linux host should be able to run Clear
Containers, as long as the host system kernel contains the necessary
features and is configured with the necessary support enabled.
.. [to do: finish this section]
Q. **"Do I need to use all of Clear Containers, or can I cherry pick parts?"**
A. You can cherry pick the parts of Clear Containers you need. Some parts
will make your life generally easier (such as the `QEMU`_ wrapper tool
``cor``) and will help insulate you from future development changes, so you
should consider which parts you need for which features. The client
side obviously can be quite flexible in its configuration depending
on the deployment environment.
Q. **"Can I use Clear Containers technology to run other VMs, not just
container style ones?"**
A. Yes, the underlying mechanisms and accelerations used for Clear
Containers can be applied to any Virtual Machine setup, not just
those that are based around a container style workflow.
.. _SR-IOV: http://www.intel.com/content/www/us/en/pci-express/pci-sig-sr-iov-primer-sr-iov-technology-paper.html
.. _QEMU: http://www.qemu.org
.. _mmu.txt: Documentation/virtual/kvm/mmu.txt
.. _Intel ARK website: http://ark.intel.com
.. _kvmtool: https://git.kernel.org/cgit/linux/kernel/git/will/kvmtool.git/
.. _rkt: https://coreos.com/rkt/
.. _Intel Clear Containers integration guide: https://clearlinux.org/documentation/gs-clear-containers-getting-started.html