diff --git a/source/ciao/ciao.rst b/source/ciao/ciao.rst deleted file mode 100644 index 2b28d3c7..00000000 --- a/source/ciao/ciao.rst +++ /dev/null @@ -1,12 +0,0 @@ -.. _ciao: - -ciao -#### - -.. toctree:: - :maxdepth: 2 - - cluster-setup-quick - cluster-setup - deployment/deployment - deployment/ubuntu \ No newline at end of file diff --git a/source/ciao/cluster-setup-quick.rst b/source/ciao/cluster-setup-quick.rst deleted file mode 100644 index 6033d7b6..00000000 --- a/source/ciao/cluster-setup-quick.rst +++ /dev/null @@ -1,11 +0,0 @@ -.. _cluster-setup-quick: - -Easy ciao development cluster setup -################################### - -For quick ciao development cluster setup, see: - - https://github.com/01org/ciao/wiki/Single-Machine-Development-Environment - -For more detailed and configurable manual setup of ciao, see -:ref:`cluster-setup`. diff --git a/source/ciao/cluster-setup.rst b/source/ciao/cluster-setup.rst deleted file mode 100644 index 84eda71c..00000000 --- a/source/ciao/cluster-setup.rst +++ /dev/null @@ -1,541 +0,0 @@ -.. contents:: - -.. _cluster-setup: - -Detailed manual ciao cluster setup -################################## - -This topic explains how to set up a cluster of machines running Clear Linux\* -OS for Intel® Architecture with the -:abbr:`Cloud Integrated Advanced Orchestrator (CIAO)`. - -While the table of contents provides links to specific points of information, -this topic is intended as an ordered workflow. Be sure to start your cluster -components in the correct order as explained below. - -Infrastructure prerequisites -============================ - -Hardware needs --------------- - -You'll need at least four machines and a switch connecting them to form your -beginning ciao cluster. The switch is assumed to be plugged directly into an -"upstream" network running a DHCP server. See the illustration below as an -example: - -.. image:: ./figures/image-blob-ciao-networking.png - -The following examples assume you have four nodes on a ``192.168.0.0/16`` -network: - -Controller node -~~~~~~~~~~~~~~~ - -* IP ``192.168.0.101`` -* Runs Controller, Scheduler, SSL Keystone. - - -Network node ("nn") -~~~~~~~~~~~~~~~~~~~ - -* IP ``192.168.0.102`` -* Runs Launcher with ``--network=nn`` option -* Has CNCI image in ``/var/lib/ciao/images``. See below for more on CNCI - image preparation. - -Compute node 1 ("cn1") -~~~~~~~~~~~~~~~~~~~~~~ - -* IP ``192.168.0.103`` -* Runs Launcher with ``--network=cn`` option -* Has workload images in ``/var/lib/ciao/images`` - -Compute node 2 ("cn2") -~~~~~~~~~~~~~~~~~~~~~~ - -* ``IP 192.168.0.104`` -* Runs Launcher with ``--network=cn option`` -* Has workload images in ``/var/lib/ciao/images`` - - -Network needs -------------- - -Our system assumes cluster nodes have full connectivity at a routed -IP level. Additionally, the network node must have access to a DHCP -server offering addresses that are routable across the cluster. - - -Node setup -========== - -Install Clear Linux OS for Intel Architecture as host on all nodes ------------------------------------------------------------------- - -Install Clear Linux OS for Intel Architecture as the host -OS on all nodes by following the instructions in the topic -:ref:`clear-host`. The current `downloadable installer images`_ -are compatible with ciao. - -After the base installation on each node, add the following additional -bundle, which adds components needed by CIAO:: - - $ swupd bundle-add cloud-control - - - -Build the CIAO software ------------------------ - -Ciao is written in the Go programming language. It requires Go 1.7 to -build. Most Linux distributions have out-of-date versions of Go in their -repositories, so you will probably need to download and install a recent -version of Go. This can be easily done by following the -`Go installation instructions `__. - -On a Linux development machine with Go language development tooling -present, use the ``go get`` tool to fetch and build ciao and its go -dependencies:: - - $ cd $GOPATH/src - $ go get -v -u github.com/01org/ciao - -The binaries will install to ``$GOPATH/bin``. You should have -``ciao-cli``, ``ciao-cert``, ``cnci_agent``, ``ciao-launcher``, -``ciao-controller``, and ``ciao-scheduler``. - -Build certificates ------------------- - -Create the ssntp-internal communications certificates -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -On your development machine, generate the certificates for each of your -roles; general instructions can be found under the `ciao-cert`_ documentation. - -When generating the certificates, pass in the IP and host name for -the host on which you will be running the scheduler in the ``-ip`` and -``-host`` arguments, respectively. The scheduler acts as the cluster -SSNTP server, and connecting clients will validate credentials matched by -those embedded in the certificates. - -Create unique certificates for each of your scheduler, compute node, network -node launchers, cnciagent, controller, and the CNCI launcher; save each with a -unique name. The names, locations, and contents (signer and role) of the -certificates are very important. The rest of this topic will consistently use -the following example filenames: - -* ``CAcert-[scheduler-node-hostname].pem``: copy to all nodes' ``/etc/pki/ciao`` and the CNCI image's ``/var/lib/ciao``. See below for more on CNCI image preparation. -* ``cert-CNAgent-localhost.pem``: copy to all compute nodes' ``/etc/pki/ciao``. -* ``cert-CNCIAgent-localhost.pem``: copy into your network node's ``/var/lib/ciao``. A script later will copy it into the CNCI appliance image. See below for more on CNCI image preparation. -* ``cert-Controller-localhost.pem``: copy into your controller node's ``/etc/pki/ciao``. -* ``cert-NetworkingAgent-localhost.pem``: copy into your network node's ``/etc/pki/ciao``. -* ``cert-Scheduler-[scheduler-node-hostname].pem``: copy into your controller node's ``/etc/pki/ciao``. - -Create the controller web certificates -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -On your development box, generate ssl certificates for the controller's https service:: - - $ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout controller_key.pem -out controller_cert.pem - -Copy the ``controller_cert.pem`` and ``controller_key.pem`` files to your -controller node. You can use the same location where you will be storing -your controller binary (``ciao-controller``). - -You'll also need to pull that certificate into your browser as noted below in -the `Starting a workload` section. - -Keystone node -------------- - -You need to run a Keystone service. General documentation on setting -up Keystone services can be found at the `OpenStack developer`_ website. -We need a few configuration points. For example::: - - $ openstack service create --name ciao compute - $ openstack user create --password hello csr - $ openstack role add --project service --user csr admin - $ openstack user create --password giveciaoatry demo - $ openstack role add --project demo --user demo user - -This adds a ciao compute service, a keystone user and project for the -controller (a.k.a. ``csr``) node, and a demo user with the password -``giveciaoatry``. - -Controller node setup ---------------------- - -The controller node will host your controller and scheduler. Certificates are assumed -to be in ``/etc/pki/ciao``, generated with the correct roles and names -as previously described. - -Scheduler -~~~~~~~~~ - -Copy in the scheduler binary from your build/development machine to any -location, then launch it first (does not require root):: - - $ ./ciao-scheduler --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-Scheduler-[scheduler-node-hostname].pem --heartbeat - -With the optional ``--heartbeat`` option, the scheduler console will -output once per-second a heartbeat message showing connected Controller -and Compute Node client statistics. It also displays a line of -information for each command or event traversing the SSNTP server. -As the sole SSNTP server in the ciao cluster, it is a key debugging point -to understand failed flows of actions/reactions across your cluster. -Launching it first means this console output helps confirm your subsequent -cluster configurations actions are indeed succeeding. - -Compute node setup ------------------- - -Each compute node needs one launcher daemon connected to the scheduler. -Certificates are assumed to be in ``/etc/pki/ciao``, generated with the -correct roles and names as previously described. - -Copy in the launcher binary from your build/development machine to any -location. - -Prepopulate the OS image cache -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Ciao has not yet integrated with an existing image server; so for -simplicity, presume a prepopulated image cache for each compute -node in ``/var/lib/ciao/images``. - -We have tested the `Fedora 23 Cloud`_, Clear Linux OS for Intel -Architecture cloud `downloadable cloud images`_, and Ubuntu* images. Each image -will be referenced very specifically by a UUID in our configuration -files, so follow the instructions here exactly. You may wish to create -the needed UUID named image files as symlinks to a more human readable -and descriptively named image files as is done in the following example:: - - $ mkdir -p /var/lib/ciao/images - $ cd /var/lib/ciao/images - -Fedora Cloud:: - - $ curl -O https://dl.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-23-20151030.x86_64.qcow2 - $ ln -s Fedora-Cloud-Base-23-20151030.x86_64.qcow2 73a86d7e-93c0-480e-9c41-ab42f69b7799 - -Clear Linux OS for Intel Architecture Cloud:: - - $ LATEST=$(curl https://download.clearlinux.org/latest) - $ curl -O https://download.clearlinux.org/image/clear-${LATEST}-cloud.img.xz - $ xz -T0 --decompress clear-${LATEST}-cloud.img.xz - $ ln -s clear-${LATEST}-cloud.img df3768da-31f5-4ba6-82f0-127a1a705169 - -Docker* images will be pulled down automatically at the time of first usage. - -Each compute node needs its ``/var/lib/ciao/images`` directory populated with -images with which you wish to test. - -Start the compute node launcher -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The launcher is run with options declaring certificates, maximum VMs -(controls when FULL is returned by a node, scaling to the resources -available on your node), server location, and compute node ("cn") -launching type. For example:: - - $ sudo ./launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-CNAgent-localhost.pem --server= --network=cn --compute-net --mgmt-net - -Optionally, add ``-logtostderr`` (more verbose with also ``-v=2``) to get -console logging output. - -The launcher runs as root because launching QEMU/KVM virtual machines -requires ``/dev/kvm`` and other restricted resource access. - -Network node setup ------------------- - -The network node hosts VMs running the :abbr:`Compute Network Concentrators -Instance (CNCI)` or the **CNCI Agent**, one for each tenant. These VMs -are automatically launched by the controller. - -Certificates are assumed to be in ``/etc/pki/ciao``, generated with the -correct roles and names as previously described. - -Pre-populate the CNCI image cache -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -This section describes how to generate a CNCI image from a vanilla -Clear Cloud qcow2 image:: - - $ cd /var/lib/ciao/images - $ curl -O https://download.clearlinux.org/demos/ciao/clear-7470-ciao-networking.img.xz - $ xz -T0 --decompress clear-7470-ciao-networking.img.xz - $ ln -s clear-7470-ciao-networking.img 4e16e743-265a-4bf2-9fd1-57ada0b28904 - $ $GOPATH/src/github.com/01org/ciao/networking/cnci_agent/scripts/update_cnci_cloud_image.sh /var/lib/ciao/images/clear-7470-ciao-networking.img /etc/pki/ciao/ - -Start the network node launcher -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The network node's launcher is run similarly to the compute node's launcher. -The primary difference is that it uses the network node ("nn") launching -type:: - - $ sudo ./ciao-launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-NetworkingAgent-localhost.pem --server= --network=nn --compute-net --mgmt-net - -Start the controller --------------------- - -Starting the Controller on the controller node is what truly activates your -cluster for use. - -#. Copy in the ciao-controller binary from your build/development machine to any - location. Certificates are assumed to be in ``/etc/pki/ciao``, generated with - the correct roles and names as previously described. - -#. Copy in the initial database table data from the ciao-controller source - (``$GOPATH/src/github.com/01org/ciao/ciao-controller`` on your - build/development) to the same directory as the ciao-controller binary. - Copying in ``*.csv`` will work if you are testing a Clear Cloud image, - Fedora image and Docker. Other images will require edits to the csv - config files. - -#. Copy in the test.yaml file from - ``$GOPATH/src/github.com/01org/ciao/ciao-controller/test.yaml``. - -The `ciao-controller workload_resources.csv`_ and the -`ciao-controller workload_template.csv`_ have four stanzas, so yours -should as well, in order to successfully run each of the four images -currently described earlier on this page (Fedora, Clear, Docker Ubuntu, -CNCI). To run other images of your choosing, follow a process similar to -the above: pre-populate OS images and edit each of these two files on -your controller node. - -If the controller is on the same physical machine as the scheduler, the -``--url`` option is optional; otherwise it refers to your scheduler -SSNTP server IP. - -In order for the ciao-controller's go code to correctly use the CA -certificate(s) generated earlier when you built your keystone server, -this certificate needs to be installed in the control node and be -part of the control node CA root. On Clear Linux OS for Intel -Architecture, this is accomplished with:: - - $ sudo mkdir /etc/ca-certs - $ sudo cp cacert.pem /etc/ca-certs - $ sudo c_hash /etc/ca-certs/cacert.pem - -Note the generated hash from the prior command and use it in the next commands:: - - $ sudo ln -s /etc/ca-certs/cacert.pem /etc/ca-certs/ - $ sudo mkdir /etc/ssl - $ sudo ln -s /etc/ca-certs/ /etc/ssl/certs - $ sudo ln -s /etc/ca-certs/cacert.pem /usr/share/ca-certs/ - -You will need to tell the controller where the keystone service is located and -pass the ciao service username and password to it. DO NOT USE -localhost for your server name; **it must be the fully qualified DNS -name of the system that is hosting the keystone service**. -An SSL-enabled Keystone is required, with additional parameters -for ciao-controller pointing at its certificates:: - - $ sudo ./ciao-controller --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-Controller-localhost.pem -identity=https://[keystone-FQDN]:35357 --username= --password= --url --httpskey=./key.pem --httpscert=./cert.pem - -Optionally add ``-logtostderr`` (more verbose with also ``-v=2``) to get -console logging output. - -Use the `ciao-cli`_ command line tool to verify that your cluster is -now up and running (NOTE: see section below for required environment -variables for `ciao-cli`_):: - - $ ciao-cli node status - $ ciao-cli node list -compute - $ ciao-cli node list -cnci - -``node status`` shows the number of nodes in your cluster, and the -status of each. - -``node list -compute`` displays a more detailed view (number of instances per node, -available resources per node, etc.). - -``node list -cnci`` provides information about the current CNCI VMs, and their statuses. - -Interacting with your cluster -============================= - -Ciao Web UI setup ------------------ - -In addition to `ciao-cli`_, a node.js-based web UI offers a means of -interacting with your cluster visually. Documentation for this is in -the `ciao-webui`_ github repository. A simple JSON configuration file -allows you to specify the webui configuration and point its back end to -your keystone and ciao-controller systems. - -Ciao CLI setup --------------- - -The `ciao-cli`_ command-line tool can be set up by exporting a set of ciao- -specific environment variables: - -* ``CIAO_CONTROLLER`` exports the ciao controller FQDN -* ``CIAO_IDENTITY`` exports the ciao keystone instance FQDN -* ``CIAO_COMPUTEPORT`` exports the ciao compute alternative port -* ``CIAO_USERNAME`` exports the ciao username -* ``CIAO_PASSWORD`` export the ciao password for ``CIAO_USERNAME`` - -For example:: - - $ cat ciao-cli-example.sh - - export CIAO_CONTROLLER=ciao-ctl.intel.com - export CIAO_IDENTITY=https://ciao-identity.intel.com:35357 - export CIAO_USERNAME=user - export CIAO_PASSWORD=ciaouser - - $ source ciao-cli-example.sh - -Defining those variables is optional. The same pieces of information -can be passed to `ciao-cli`_ through the various command line options. -The command line options will take precedence over the ciao environment -variables and override them: - -* ``CIAO_CONTROLLER`` can be defined by the ``--controller`` option -* ``CIAO_IDENTITY`` can be defined by the ``--identity`` option -* ``CIAO_COMPUTEPORT`` can be defined by the ``--computeport`` option -* ``CIAO_USERNAME`` can be defined by the ``--username`` option -* ``CIAO_PASSWORD`` can be defined by the ``--password`` option - - -Start a workload -================ - -As a valid user, the `ciao-cli` tool allows you to start a workload. - -First, you may want to know which workloads are available:: - - $ ciao-cli workload list - -Then you can launch one or more workloads:: - - $ ciao-cli instance add -workload -instances - -And you can monitor all your instances statuses (``pending`` or ``running``):: - - $ ciao-cli instance list - -Performance data can be obtained (optionally) by adding a specific label -to all your instances:: - - $ ciao-cli instance add -label -workload -instances - -And eventually fetch the performance data:: - - $ ciao-cli trace show -label - -You will also see activity related to this launch across your cluster -components if you have consoles open and logging to standard output as -described above. - - -Access your workload -==================== - -Once your workload is up, you need to know its IP address and assigned -port, you can find it via ciao-cli:: - - $ ciao-cli instance list - -Then look for the column "SSH IP", there is the IP assigned to your -workload, next to it, you will see the column SSH PORT; that's the -port that you will use to access a specific workload using your -private key defined in the cloud-init configuration and the demo -user name ``demouser``:: - - $ ssh -p -i demouser@ - - -Reset your cluster -================== - -First you should delete all instances with the `ciao-cli`_ command line -tool:: - - $ ciao-cli instance delete -all - -On your scheduler node, run the following command:: - - $ sudo killall -w -9 qemu-system-x86_64 - -On your controller node, go to the directory where you ran the -ciao-controller binary and run the following commands:: - - $ sudo killall -w -9 ciao-controller - $ sudo rm $HOME/bin/ciao-controller.db /tmp/ciao-controller-stats.db - -On the node running your keystone VM, run the following command:: - - $ sudo killall -w -9 qemu-system-x86_64 - -On the network node, run the following commands:: - - $ sudo ./launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-NetworkingAgent-localhost.pem --server= --network=nn --compute-net --mgmt-net --hard-reset - $ sudo killall -9 qemu-system-x86_64 - $ sudo rm -rf /var/lib/ciao/instances/ - $ sudo reboot - -If you were unable to successfully delete all workload VM instances -through the UI, then on each compute node run these commands:: - - $ sudo ./launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-CNAgent-localhost.pem --server= --network=cn --compute-net --mgmt-net --hard-reset - $ sudo killall -9 qemu-system-x86_64 - $ sudo docker rm $(sudo docker ps -qa) - $ sudo docker network rm $(sudo docker network ls -q -f "type=custom") - $ sudo rm -rf /var/lib/ciao/instances/ - $ sudo reboot - -Restart your scheduler, network node launcher, compute node launcher, -and controller. - -Debug tips -========== - -For general debugging, you can: - -* Reset you cluster. -* Pull in updated go binaries. -* Enable verbose console logging with ``-logtostderr -v=2`` on the go - binaries' command lines. -* Reduce your tenants to one (specifically the one with no limits). -* Launch fewer VMs in a herd. A small Intel NUC with 16GB of RAM can handle as many as - 50-100 2vcpu 218MB RAM VMs starting at once per compute node. Larger dual-socket - many threaded CPUs with hundreds of GB RAM Haswell-EP servers can handle as many as - 500 such VMs starting at once per compute node. -* Tweak the launcher to enable remote access: go get with ``--tags=debug`` to enable - a netcat based console redirection for each VM. The launcher console verbose output - will indicate per VM how to connect to the console. For example:: - - $ netcat 192.168.0.102 6309 - -* Ssh into the compute node(s) by IP, looking at top, df, ps, ip a, ip r, netstat -a, etc. -* Ssh into the CNCI(s) by IP, looking at top, df, ps, ip a, ip r, netstat -a, etc. -* Ssh into the workload instance VMs via CNCI IP and port redirection. Each VM will be - at a port composed from the VM's IP address added to 33000. For example:: - - 33000+ip[2]<<8+ip[3] - - The VM IP is available in the `ciao-cli`_. -* Instance credentials for netcat or ssh connectivity depend on the contents of - the cloud-init configuration used by ciao-controller for the workload. - -Please contact our `mailing list`_ for more help with initial bringup and -testing. - -.. _ciao-controller workload_resources.csv: https://github.com/01org/ciao/blob/master/ciao-controller/workload_resources.csv -.. _ciao-controller workload_template.csv: https://github.com/01org/ciao/blob/master/ciao-controller/workload_template.csv -.. _downloadable installer images: https://download.clearlinux.org/image -.. _downloadable cloud images: https://download.clearlinux.org/image -.. _Fedora 23 Cloud: https://download.fedoraproject.org/pub/fedora/linux/releases/23/Cloud/x86_64/Images/Fedora-Cloud-Base-23-20151030.x86_64.qcow2 -.. _Openstack developer: http://docs.openstack.org/developer/keystone/setup.html -.. _go: https://golang.org/doc/articles/go_command.html -.. _ciao-cert: https://github.com/01org/ciao/blob/master/ssntp/ciao-cert/README.md -.. _CNCI Agent: https://github.com/01org/ciao/tree/master/networking/cnci_agent -.. _mailing list: https://lists.clearlinux.org/mailman/listinfo/ciao-devel -.. _ciao-cli: https://github.com/01org/ciao/tree/master/ciao-cli -.. _ciao-webui: https://github.com/01org/ciao-webui diff --git a/source/ciao/deployment/deployment.rst b/source/ciao/deployment/deployment.rst deleted file mode 100644 index 3c4f6915..00000000 --- a/source/ciao/deployment/deployment.rst +++ /dev/null @@ -1,165 +0,0 @@ -.. contents:: - -.. _deployment: - -Deploying ciao via automation -############################# - -Cloud Integrated Advanced Orchestrator (``ciao``) is a new workload -scheduler designed to address limitations of current cloud OS projects. -Ciao provides a lightweight, fully TLS-based minimal config, is -workload-agnostic, easy to update, offers an optimized-for-speed -scheduler, and is currently optimized for OpenStack*. - -For more information, see https://clearlinux.org/ciao. - -Environment -=========== - -For this example, we'll use a total of five nodes: - - A deployment machine which will be used to run the playbooks. - - A `controller`_ node which will be used to communicate with Keystone. - - Two `compute nodes`_, which will spawn the VMs and containers. - - A `network node`_ which will handle the networking for the workloads. - -.. note:: - - The deployment machine is not a necessary part of the cluster; it could be - the sysadmin's computer or a CI/CD server. - -Prerequisites -============= -Ansible* uses :command:`ssh` to run commands on the remote nodes. In order to do -that, configure a user for passwordless SSH connections from the deployment -container to the cluster nodes. This user must also have passwordless sudo -privileges on the cluster nodes. - - -Configure your cluster setup -============================ -Ensure the following packages are installed before running the deployment -node: - - * ansible version 2.2.1.0 or higher - * openstackclient - * python-netaddr - * gcc - * go version 1.7 or higher - * git - -Install all of the needed packages with the following commands: - -.. code-block:: console - - # swupd bundle-add sysadmin-hostmgmt c-basic go-basic - # pip2 install python-keystoneclient - - -You will need to download the ciao example deployment as follows: - -.. code-block:: console - - $ git clone https://github.com/01org/ciao.git - -Once you're cloned the repo, continue working in the -`ciao/_DeploymentAndDistroPackaging/ansible/` directory - -.. code-block:: console - - # cd $(pwd)/ciao/_DeploymentAndDistroPackaging/ansible/ - -Next, edit the configuration files for the cluster: - - * The `hosts`_ file is the hosts inventory file which contains the IP - addresses/FQDN of your nodes, grouped under the roles they will serve. - - * The `groups_vars/all`_ file contains variables that will be applied - to your ciao setup. The mandatory variables are already there; be - sure to change the values accordingly to fit your environment. - - * The ``ciao_guest_key`` value in :file:`groups_var/all` is the key to be - used to connect to the VMs created by ciao; you can use the - ``ssh-keygen`` command to create one. - -A full list of available variables can be found in the -:file:`defaults/main.yml` file of each role at -https://github.com/01org/ciao/tree/master/_DeploymentAndDistroPackaging/ansible/roles. - - -Run the playbook -================ - -After the variables and the :file:`hosts` file are configured, start the -deployment with the following commands: - -.. code-block:: console - - # cd ciao/_DeploymentAndDistroPackaging/ansible/ - - # ansible-playbook -i hosts ciao.yml \ - --private-key=~/.ssh/id_rsa \ - --user= - -.. note:: - - Note: The playbook will create the following files in the current folder of - the machine running the playbooks. - - * ``./certificates``: This directory contains the certificates - that where created and copied to the cluster nodes. - - * ``./images``: This directory contains the images used by the - ciao cluster (fedora, clearlinux, cnci, ovmf.fd). - - * ``./ciaorc``: This file contains environment variables needed - by ciao cli to authenticate to the ciao cluster. - - * ``./openrc``: This file contains environment variables needed by - openstack cli to authenticate with the ciao cluster. - -Verify -====== -After Ansible is done with the setup, you can verify the cluster is ready -by running the following command on the controller node. Change the **username**, -**password**, **controller**, and **identity** values to match your setup, as -was specified in the ``groups_var/all`` file: - -.. code-block:: console - - # ciao-cli -identity=https://ciao-controller.example.com:35357 -username ciao -password ciaoUserPassword -controller=ciao-controller.example.com node status - Total Nodes 3 - Ready 0 - Full 3 - Offline 0 - Maintenance 0 - -You could also take a look at the ``./ciaorc`` file created on your -deployment node, which contains the following environment variables: - -.. code-block:: console - - # cat ciaorc - export CIAO_CONTROLLER=ciao-controller.example.com - export CIAO_IDENTITY=https://ciao-controller.example.com:35357 - export CIAO_USERNAME=ciao - export CIAO_PASSWORD=ciaoUserPassword - -Then you could verify with the following command: - -.. code-block:: console - - # source ciaorc - # ciao-cli node status - Total Nodes 3 - Ready 0 - Full 3 - Offline 0 - Maintenance 0 - -.. _controller: https://github.com/01org/ciao/tree/master/ciao-controller -.. _compute nodes: https://github.com/01org/ciao/tree/master/ciao-launcher -.. _network node: https://github.com/01org/ciao/tree/master/ciao-launcher -.. _hosts: https://github.com/01org/ciao/blob/master/_DeploymentAndDistroPackaging/ansible/hosts -.. _groups_vars/all: https://github.com/01org/ciao/blob/master/_DeploymentAndDistroPackaging/ansible/group_vars/all -.. _CNCI image: https://github.com/01org/ciao/tree/master/networking/ciao-cnci-agent#cnci-agent -.. _Docker* documentation: https://docs.docker.com/engine/reference/commandline/run/ diff --git a/source/ciao/deployment/ubuntu.rst b/source/ciao/deployment/ubuntu.rst deleted file mode 100644 index f9e29b1d..00000000 --- a/source/ciao/deployment/ubuntu.rst +++ /dev/null @@ -1,218 +0,0 @@ -.. contents:: - -.. _ubuntu: - -Deploying ciao via automation -- Ubuntu 16.04 -############################################# - -Cloud Integrated Advanced Orchestrator (``ciao``) is a new workload -scheduler designed to address limitations of current cloud OS projects. -Ciao provides a lightweight, fully TLS-based minimal config, is -workload-agnostic, easy to update, offers an optimized-for-speed -scheduler, and is currently optimized for OpenStack\*. - -For more information, see https://clearlinux.org/ciao. - -Environment -=========== - -For this example, we'll use a total of five nodes: - - A deployment machine which will be used to run the playbooks. - - A `controller`_ node which will be used to communicate with Keystone. - - Two `compute nodes`_, which will spawn the VMs and containers. - - A `network node`_ which will handle the networking for the workloads. - - Note: The deployment machine is not a necessary part of the cluster; it - could be the sysadmin's computer or a CI/CD server. - -Prerequisites -============= - -Ansible* uses :command:`ssh` to run commands on the remote nodes. In -order to do that, the nodes must be configured to allow passwordless SSH -connections from the deployment machine to the cluster nodes. The user -should also have sudo privileges on the cluster nodes. - -The deployment machine needs:: - - Python 2.7 - Ansible >= 2.1 - Golang >= 1.7 - qemu-utils - python-docker - python-openstackclient - Ansible ciao roles from ansible-galaxy - -The managed nodes need:: - - python 2.6 >= - - -Proxy -===== - -If you are running behind a proxy, the following additional steps are needed: - -In the deployment node ----------------------- - -Make sure apt can run. Edit ``/etc/apt/apt.conf`` (to install ansible and -dependencies), appending it with:: - - Acquire::http::Proxy "http://yourproxyaddress:proxyport"; - Install the CIAO roles from ansible-galaxy: - $ sudo -E ansible-galaxy -r requirements --ignore-certs - -In the managed nodes --------------------- - -Make sure apt can run. Edit ``/etc/apt/apt.conf``, appending it with:: - - Acquire::http::Proxy "http://yourproxyaddress:proxyport"; - -Install Docker and make sure you can pull images. Edit -``/etc/systemd/system/docker.service.d/http-proxy.conf``, appending it -with:: - - [Service] - Environment="HTTP_PROXY=http://hostname:port/" - Environment="HTTPS_PROXY=http://hostname:port/" - Environment=”NO_PROXY=localhost,127.0.0.1,.example.com” - -Note: replace the hostname and port of your proxy server and append your local domain name. - -Reload and restart the docker daemon:: - - $ sudo systemctl daemon-reload && sudo systemctl restart docker - - (OPTIONAL): You can download the docker images used in the CIAO deployment. - $ sudo docker pull clearlinux/keystone - $ sudo docker pull clearlinux/ciao-webui - - -Install the software -==================== - -Install ansible from ubuntu packages if the package version is >= 2.1.0; -otherwise, install it from pip (``pip install ansible``). - -Install Dependencies in the deployment machine ----------------------------------------------- - -:: - - python-docker (apt-get) or docker-py (pip) - python-openstackclient (apt-get) - qemu-utils (apt-get) - -Download the ``clear-config-management`` project from github:: - - $ git clone https://github.com/clearlinux/clear-config-management.git - - -Create the playbook -=================== - -The ``clear-config-management`` project includes some sample playbooks that -you may use and customize for your own needs. Start by making a copy of the -sample playbook into your home folder:: - - # cp -r clear-config-management/examples/ciao ~/ - - -Note: These files are also hosted in github - -The relevant files in the playbook are the following: - - * The `ciao.yml`_ file is the master playbook file and includes a playbook - for each component of the cluster. - - * The `hosts`_ file is the hosts inventory file and contains the IP - addresses/FQDN of your nodes, grouped under the roles they will serve - - * The `groups_vars/all`_ file contains variables that will be applied - to your ciao setup. The mandatory variables are already there; be - sure to change the values accordingly to fit your environment - - * The ``ciao_guest_key`` value in :file:`groups_var/all` is the key - to be used to connect to the VMs created by ciao; you can use the - ``ssh-keygen`` command to create one. - -A full list of available variables can be found in the :file:`defaults/main.yml` -file of each role at https://github.com/clearlinux/clear-config-management/tree/master/roles - - -Install the required ansible-roles -================================== - -.. code-block:: console - - # ansible-galaxy install -r requirements.yml - - -Run the playbook -================ - -Once you have your variables and hosts file configured, the deployment can -be started with the following command: - -.. code-block:: console - - $ ansible-playbook -i hosts ciao.yml --private-key= - -Note: If you want the latest CIAO changes, change the ``ciao_dev`` -variable to ``True`` in the :file:`group_vars/all` file In the -``clear-config-management`` project. - - -Verify -====== - -After ansible is done with the setup, you can verify the cluster is ready -by running the following command on the controller node. Change the **username**, -**password**, **controller**, and **identity** values to match your setup, as -was specified in the ``groups_var/all`` file: - -.. code-block:: console - - # ciao-cli -identity=https://ciao-controller.example.com:35357 \ - -username admin \ - -password secret \ - -controller=ciao-controller.example.com - # node status - Total Nodes 3 - Ready 0 - Full 3 - Offline 0 - Maintenance 0 - -You could also take a look at the :file:`./ciaorc` file created on your -deployment node, which contains the following environment variables: - -.. code-block:: console - - # cat ciaorc - export CIAO_CONTROLLER=ciao-controller.example.com - export CIAO_IDENTITY=https://ciao-controller.example.com:35357 - export CIAO_USERNAME=csr - export CIAO_PASSWORD=secret - -then you could verify with the following command: - -.. code-block:: console - - # source ciaorc - # ciao-cli node status - Total Nodes 3 - Ready 0 - Full 3 - Offline 0 - Maintenance 0 - -.. _controller: https://github.com/01org/ciao/tree/master/ciao-controller -.. _compute nodes: https://github.com/01org/ciao/tree/master/ciao-launcher -.. _network node: https://github.com/01org/ciao/tree/master/ciao-launcher -.. _ciao.yml: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/ciao.yml -.. _hosts: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/hosts -.. _groups_vars/all: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/group_vars/all -.. _github: https://github.com/clearlinux/clear-config-management/tree/master/examples/ciao diff --git a/source/ciao/figures/image-blob-ciao-networking.png b/source/ciao/figures/image-blob-ciao-networking.png deleted file mode 100644 index 77f3d833..00000000 Binary files a/source/ciao/figures/image-blob-ciao-networking.png and /dev/null differ diff --git a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst index cb694681..9e9277d6 100644 --- a/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst +++ b/source/clear-linux/get-started/bare-metal-install/bare-metal-install.rst @@ -1,286 +1,319 @@ -.. _bare-metal-install: - -Install Clear Linux host OS on bare metal -######################################### - -This instruction set will guide you through the automatic installation of |CL| -on bare metal using a bootable USB drive. - -.. include:: ../compatibility-check.rst - :Start-after: compatibility-check: - -.. include:: ../bootable-usb/bootable-usb-linux.rst - :Start-after: bootable-usb-linux: - :end-before: download-cl-image - -.. include:: ../bootable-usb/bootable-usb-linux.rst - :Start-after: download-cl-image: - :end-before: verify-checksum - -.. include:: ../bootable-usb/bootable-usb-linux.rst - :Start-after: verify-checksum: - :end-before: copy-usb-linux - -.. include:: ../bootable-usb/bootable-usb-linux.rst - :Start-after: copy-usb-linux: - :end-before: usb-next - -.. _install-on-target: - -Install Clear Linux on your target system -***************************************** - -The USB drive that was created in the previous step has been formatted as a -UEFI boot device. Our target system has a hard drive installed containing a -single primary partition. The target system needs a wired Internet connection -with DHCP. - -Follow these steps to install |CL| on the target system: - -#. Insert the USB drive into an available USB slot. - -#. Power on the system. - -#. Open the system BIOS setup menu, normally by pressing the F2 key. Your - BIOS setup menu entry point may vary. - -#. In the setup menu, enable the UEFI boot and set the USB drive as the first - option in the device boot order. - -#. Save these settings and exit. - -#. Reboot the target system and the |CL| Installer menu will start. - - .. note:: - - Use the arrow keys, space bar, and enter key to navigate the menu of the - |CL| Installer. - -#. In this tutorial, we will enable telemetrics and select the `Automatic` - installation type. - - The primary drive to install |CL| onto is :file:`/dev/sdb` since the - target system identifies the USB drive as :file:`/dev/sda`. - -#. Follow the instructions to begin the installation. - - The installation is completed in the following stages: - - .. code-block:: console - - Reading configuration - Validating configuration - Creating partitions - Creating file systems - Setting up mount points - Starting swupd. May take several minutes - Cleaning up - Successful installation - - Once complete, an `` prompt appears in the dialog box, letting you know - that you have successfully installed |CL| onto your system. - -#. Press enter once the prompt appears. The following dialog box takes - its place: - - .. code-block:: console - - Successful installation, the system will be rebooted - - - -#. Press enter, remove the USB drive from the system, and the system will - reboot running |CL|. - -.. _initial-setup: - -Clear Linux initial setup after installation -******************************************** - -Your newly installed |CL| boots on your target system and presents a full -screen console requesting `Login:`. |CL| is designed to install with minimal -software overhead. Therefore, some housekeeping and package installations -must occur before you have a full-featured |CL| operating system. - -Set up your root and user accounts -================================== - -#. At the initial login prompt, enter: root - -#. Once you are prompted, enter a new password - -#. Re-enter the password to verify it. - -You have set your root password and are logged in with root privileges. - -The next step is to create a new user and set a password for -that user: - -.. code-block:: console - - useradd - passwd - -Replace with the name of the user account you want to create. - -Remain logged in as the root user because there are some other things to do -before we can fully enable your new user space. - -Software installation and updates -================================= - -|CL| has a unique application and architecture to add and update applications -and to perform system updates called software update utility or `swupd`. -Software applications are installed as bundles using the command -:command:`bundle-add`. - -Next, we should install some useful applications using the software update -utility. The `os-clr-on-clr` bundle installs the vast majority of -applications useful to a system administrator or a developer. The bundle -contains other bundles such as `sysadmin-basic`, `editors`, `c-basic`, `dev- -utils-dev`, and other useful packages. - -.. code-block:: console - - swupd bundle-add os-clr-on-clr - -.. note:: - - The image we installed may not be the latest version of |CL| available on - the server. However, whenever the command - :command:`swupd bundle-add ` runs, the OS is updated to the latest - available version. Our website provides more `information about swupd`_. - -We provide the full list of bundles and packages installed with the -`os-clr-on-clr`_ bundle. Additionally, we have listed -`all Clear Linux bundles`_, active or deprecated. Click any bundle on the -list to view the manifest of the bundle. - -Finish setting up your new user -=============================== - -Before logging off as root and logging into your new user account, we must -enable the :command:`sudo` command for your new ``. - -To be able to execute all applications with root privileges, we must add the -`` to the `wheel group`_ and enable the wheel group in the -:file:`/etc/sudoers` file. - -#. To add `` to the wheel group, enter the following command: - - .. code-block:: console - - usermod -G wheel -a - -#. To open the :file:`/etc/sudoers` file, enter the following command: - - .. code-block:: console - - vi /etc/sudoers - - .. note:: - - Normally, we would use the visudo script to edit the :file:`/etc/sudoers` - file to safely modify the contents of the file. In this instance, the file - does not exist yet. Therefore, we create the initial instance of the file. - -#. In the vi\* editor window, press the :kbd:`o` to open a new line. - -#. Add the following line to the file: - - .. code-block:: console - - %wheel ALL=(ALL) ALL - -#. To save the changes to the file and exit vi, press the :kbd:`ESC` key - followed by the :kbd:`:` and :kbd:`x` keys. - - .. important:: - - Creating the file logged as the root user keeps the permissions of the - file with the root user. - - Now, we can log out of root and into our new . - -#. To log off as root, enter :command:`exit`. - - The command should bring you back to the `Login:` prompt. - -#. Enter your new `` and the password you created earlier. - - You should now be in the home directory of ``. The bundle - `os-clr-on-clr`_ contains the majority of applications that a developer or - system administrator would want but it does not include a graphical user - interface. The `desktop` bundle includes the Gnome Desktop Manager and - additional supporting applications. - -Install a GUI -============= - -#. To test the :command:`sudo` command and ensure we set it up correctly, we - can install the Gnome Desktop Manager (gdm) and start it. - -#. To install Gnome using swupd, enter the following command: - - .. code-block:: console - - sudo swupd bundle-add desktop - -#. To start the Gnome Desktop Manager, enter the following command: - - .. code-block:: console - - systemctl start gdm - - You will be prompted to authenticate your user. Enter the password for - `` and the Gnome Desktop should start as shown in figure 1: - - .. figure:: figures/gnomedt.png - :alt: Gnome Desktop - - Figure 1: :guilabel:`Gnome Desktop` - -#. To start the Gnome Desktop each time you start your system, enter - the following command: - - .. code-block:: console - - systemctl enable gdm - -**Congratulations!** - -You have successfully installed |CL| on a bare metal system. - -Additionally, you performed the following basic setup for your system: - -* Setup of a root user. -* Updated the OS to its most current version using `swupd`. -* Installed the most common applications for system administrators and - developers using bundles. -* Setup of a new user. -* Setup of `sudo` privileges for that new user. -* Installed a GUI using those `sudo` privileges. - -Next steps -********** - -With your system now running |CL| many paths are open for you. - -Visit our :ref:`tutorials` page for examples on using your |CL| -system. - -.. _`information about swupd`: - https://clearlinux.org/features/software-update - -.. _`os-clr-on-clr`: - https://github.com/clearlinux/clr-bundles/blob/master/bundles/os-clr-on-clr - -.. _`all Clear Linux bundles`: - https://github.com/clearlinux/clr-bundles/tree/master/bundles - -.. _`wheel group`: - https://en.wikipedia.org/wiki/Wheel_(Unix_term) - -.. _image: https://download.clearlinux.org/image - -.. _releases: https://download.clearlinux.org/releases +.. _bare-metal-install: + +Install Clear Linux OS on bare metal +#################################### + +These instructions guide you through the installation of |CLOSIA| +on bare metal using a bootable USB drive. + +.. include:: ../compatibility-check.rst + :Start-after: compatibility-check: + +.. include:: ../bootable-usb/bootable-usb-linux.rst + :Start-after: bootable-usb-linux: + :end-before: download-cl-image + +.. include:: ../bootable-usb/bootable-usb-linux.rst + :Start-after: download-cl-image: + :end-before: verify-checksum + +.. include:: ../bootable-usb/bootable-usb-linux.rst + :Start-after: verify-checksum: + :end-before: copy-usb-linux + +.. include:: ../bootable-usb/bootable-usb-linux.rst + :Start-after: copy-usb-linux: + :end-before: usb-next + +.. _install-on-target: + +Install Clear Linux on your target system +***************************************** + +We formatted the previously created USB drive as a UEFI boot device. Our +target system has a hard drive installed containing a single primary +partition. The target system needs a wired Internet connection with DHCP. + +Follow these steps to install |CL| on the target system: + +#. Insert the USB drive into an available USB slot. + +#. Power on the system. + +#. Open the system BIOS setup menu, normally by pressing the :kbd:`F2` key. + Your BIOS setup menu entry point may vary. + +#. In the setup menu, enable the UEFI boot and set the USB drive as the first + option in the device boot order. + +#. Save these settings and exit. + +#. Reboot the target system. + +#. The |CL| Installer menu will start as shown in Figure 1. + Select :guilabel:`Clear Linux OS for Intel Architecture` and press the + :kbd:`Enter` key or wait five seconds to automatically select it. + + .. figure:: figures/bare-metal-install-1.png + :scale: 50 % + :alt: Clear Linux boot menu + + Figure 1: :guilabel:`Clear Linux boot menu` + +#. This will take you into the :guilabel:`Clear Linux OS for Intel + Architecture Installer` menu as shown in figure 2 and explains how to + navigate through the |CL| installer setup menus. + + .. figure:: figures/bare-metal-install-2.png + :scale: 50 % + :alt: Clear Linux OS for Intel Architecture Installer + + Figure 2: :guilabel:`Clear Linux OS for Intel Architecture Installer` + + Press the :kbd:`Enter` key. + +#. The :guilabel:`Keyboard selection` menu shown in figure 3 allows you to set + up the keyboard layout that you will be using to navigate within the |CL| + installer setup menus. + + .. figure:: figures/bare-metal-install-3.png + :scale: 50 % + :alt: Keyboard Selection + + Figure 3: :guilabel:`Keyboard Selection` + + For this guide we will select :menuselection:`Keyboard Selection --> + < * us >` for the keyboard mapping, which should already be highlighted. + Press the :kbd:`Enter` key to continue to the next menu. + +Network requirements +==================== + +The :guilabel:`Network Requirements` menu, the first step of the |CL| +installer setup process, will attempt to connect to the |CL| update server +where the installer image is located. Once the connection to the |CL| update +server is established, you will see a screen similar to the one shown in +figure 4: + +.. figure:: figures/bare-metal-install-4.png + :scale: 50 % + :alt: Network Requirements + + Figure 4: :guilabel:`Network Requirements` + +If you need to configure any :guilabel:`Proxy Settings` to gain access to the +update server, enter the appropriate address and port of your proxy server in +the :guilabel:`HTTPS proxy:` field. Select the :guilabel:`< Set proxy +configuration >` button and press :kbd:`Enter`. You will then see the +connection to the update server established. + +Optionally, set up a :guilabel:`static IP configuration` to your |CL| +installer image. Enter the required information in the :guilabel:`Interface`, +:guilabel:`IP address`, :guilabel:`Subnet mask`, :guilabel:`Gateway` and +:guilabel:`DNS` fields and then select the :guilabel:`< Set static IP +configuration >` button and press the :kbd:`Enter` key. + +The information displayed in the lower right quadrant of the screen shows the +current IP configuration for the |CL| update server where the installer image +is located. + +.. note:: + + If you are having difficulty establishing a connection to the update server + and you see the message :guilabel:`none detected, install will fail`, you + can press the :kbd:`Tab` key to highlight the :guilabel:`< Refresh >` + button and press :kbd:`Enter` to attempt to reconnect to the |CL| update + server. If this fails to establish a connection after multiple attempts, + reboot your system and return to this step. + +Once the connection to the |CL| udpate server is established, use the +:kbd:`Tab` key to advance to the :guilabel:`< Next >` button and press +:kbd:`Enter` to advance to the next |CL| installer setup menu. + +Choose Clear Linux installer action +=================================== + +The :guilabel:`Choose Action` menu is where you can choose to install, repair, +open a shell, or exit the |CL| installer. This menu is shown in figure 5: + +.. figure:: figures/bare-metal-install-5.png + :scale: 50 % + :alt: Choose Action + + Figure 5: :guilabel:`Choose Action` + +#. Select the :menuselection:`Choose action --> Install` menu item to continue + the installation process. + + The :menuselection:`Choose action --> Repair` menu option will run the + :command:`swupd --fix` command to correct any issues found with the system + software that has already been installed on your system and correct any + issues found by overwriting the incorrect file content, adding missing + files, fixing permissions and any additional changes required to return the + file to it's original content and permissions. + + The :menuselection:`Choose action --> Shell` menu item opens a terminal + session on your system as the root user and you will be able to manage your + system from this console. When you are finished, type :command:`exit` to + return to the :guilabel:`Choose Action` menu. + + The :menuselection:`Choose action --> Exit` menu option terminates the |CL| + installation process and the system will shut down. + +#. You will be prompted to join the :guilabel:`Stability Enhancement Program` + as shown in figure 6. Press the :kbd:`Spacebar` or :kbd:`Enter` key while + the cursor is in the :guilabel:`[ ] Yes.` button to enable this + functionality and then select the :guilabel:`< Next >` button to advance to + the next menu item. + + .. figure:: figures/bare-metal-install-6.png + :scale: 50 % + :alt: Stability Enhancement Program + + Figure 6: :guilabel:`Stability Enhancement Program` + + If you choose not to enable this functionality during this step, you can + install the ``telemetrics`` software bundle at a later time. As stated in + the menu, this feature only collects anonymous information about your + system to help improve system stability and no personally identifiable + information is collected. Please visit our website to + `learn more about telemetry.`_ + +Choose Clear Linux installation type +************************************ + +Figure 7 shows the next step of the |CL| installer: +:guilabel:`Choose installation Type`. Chose whether to install |CL| +**automatically** or **manually**. To end the installer process and shut down +the system, select the :guilabel:`< Exit >` button. + +.. figure:: figures/bare-metal-install-7.png + :scale: 50 % + :alt: Choose installation type + + Figure 7: :guilabel:`Choose installation Type` + +If you select :guilabel:`< Automatic >` as the installation type, the |CL| +Installer will add the minimum amount of functionality required for a fully +functional |CL| system. You will not be able to modify the disk layout, add +a user or any other tasks that the manual installation process will allow. + +With the :guilabel:`< Manual(Advanced) >` option, you can do the following +additional tasks during |CL| Installer setup: + +* Modify the disk layout using the cgdisk utility. +* Add additional command-line parameters to the kernel. +* Create a hostname for your system. +* Create an administrative user. +* Add additional software bundles to enhance the functionality of your initial + |CL| installation. +* Optionally, set up a static IP address for your system. + +If you want to perform any of these additional tasks, select the +:guilabel:`< Manual(Advanced) >` menu item and follow the steps in our +:ref:`bare-metal-manual-install` to complete the |CL| manual installation +process. Otherwise, you can follow the |CL| automatic installation steps. + +Clear Linux automatic installation +********************************** + +To install the minimum components for your |CL| implementation, select the +:guilabel:`< Automatic >` menu item shown in figure 7 and press the +:kbd:`Enter` key. + +The :guilabel:`Choose target device for installation` screen shown in figure 8 +appears. Move the cursor to the desired target and press the :kbd:`Enter` +key. + +.. figure:: figures/bare-metal-install-8.png + :scale: 50 % + :alt: Choose target device for installation + + Figure 8: :guilabel:`Choose target device for installation` + +In this example we selected the single primary partition from our hard drive. + +With all the |CL| installer setup information gathered for the automatic +installation option, the |CL| Installer prompts you to begin the actual +installation as shown in figure 9. + +.. figure:: figures/bare-metal-install-9.png + :scale: 50 % + :alt: Begin installation + + Figure 9: :guilabel:`Begin installation` + +When you are satisfied with the information you have entered, select the +:guilabel:`< Yes >` button and press :kbd:`Enter` to begin installing |CL|. + +|CL| Installation begins and each step shows its status as it progresses +through the automated installation process. + +Once all steps have completed, you will see the `:guilabel:`Successful +installation` status message and the :guilabel:`< Ok >` button is highlighted +as shown in figure 10. Press the :kbd:`Enter` key to continue. + +.. figure:: figures/bare-metal-install-10.png + :scale: 50 % + :alt: Installation complete + + Figure 10: :guilabel:`Installation complete` + +Figure 11 shows the installer's final screen prompting you that the +installation completed successfully and the system will reboot. Press the +:kbd:`Enter` key and remove the USB media while the system restarts. + +.. figure:: figures/bare-metal-install-11.png + :scale: 50 % + :alt: Successful installation + + Figure 11: :guilabel:`Successful Installation` + +Set up your root account +======================== + +Once the |CL| installation is complete and the system boots, a full screen +console requests your login: as shown in figure 12: + +.. figure:: figures/bare-metal-install-12.png + :scale: 50 % + :alt: Login screen + + Figure 12: :guilabel:`Login screen` + +#. At the initial login prompt, enter: ``root`` + +#. Once prompted, enter a new password. + +#. Re-enter the password to verify it. + +You have now set your root password and are logged in with root privileges. + +**Congratulations!** + +You have successfully installed |CL| on a bare metal system using the +automatic installation method and set the password for the ``root`` user. + +The automatic installation of |CL| is designed to install with minimal +software overhead. Therefore, some housekeeping and package installations +could be needed before you can take full advantage of the |CL| operating +system. These instructions are captured in the :ref:`enable-user-space`. + +* Create a new user +* Update the OS to its most current version using `swupd`. +* Install the most common applications for system administrators and + developers using bundles. +* Setup a new user. +* Setup `sudo` privileges for that new user. +* Install a GUI using those `sudo` privileges. + + +.. _`information about stateless`: + https://clearlinux.org/features/stateless + +.. _`learn more about telemetry.`: + https://clearlinux.org/features/telemetry + +.. _`NUC6i5SYH product page`: + http://www.intel.com/content/www/us/en/nuc/nuc-kit-nuc6i5syh.html + diff --git a/source/clear-linux/get-started/bare-metal-install/bare-metal-manual-install.rst b/source/clear-linux/get-started/bare-metal-install/bare-metal-manual-install.rst new file mode 100644 index 00000000..c3b855a2 --- /dev/null +++ b/source/clear-linux/get-started/bare-metal-install/bare-metal-manual-install.rst @@ -0,0 +1,282 @@ +.. _bare-metal-manual-install: + +Bare metal manual installation guide +#################################### + +This section contains the steps for a |CL| manual installation. It picks up +where the :ref:`bare-metal-install` left off: + +Select the :guilabel:`< Manual(Advanced) >` menu option as seen in figure +1, to do the following additional tasks during the |CL| Installer setup: + +* :ref:`Modify the disk layout using the cgdisk utility` +* Add additional command-line parameters to the kernel +* Create a hostname for your system +* Create an administrative user +* Add additional software bundles to enhance the functionality of your initial + |CL| installation +* Optionally set up a static IP address for your system + +Begin the manual installation process +************************************* + +#. Move the cursor to the :guilabel:`< Manual(Advanced) >` menu item + and press the :kbd:`Enter` key. + + .. figure:: figures/bare-metal-manual-install-1.png + :scale: 50 % + :alt: Choose installation Type + + Figure 1: :guilabel:`Choose installation Type` + +#. The :guilabel:`Choose partitioning method` screen is shown in figure 2. + + .. figure:: figures/bare-metal-manual-install-2.png + :scale: 50 % + :alt: Choose partitioning method + + Figure 2: :guilabel:`Choose partitioning method` + + If your hard drive has already been partitioned for a Linux system, you can + select the :guilabel:`< Use default partition and mount scheme on target + device >` menu item and press :kbd:`Enter` to move to the next step of the + installer setup process. + + .. _cgdisk-manual-setup: + + Otherwise, to set up your hard drive partitions manually before moving to + the next step, select the :guilabel:`< Manually configure mounts and + partitions >` menu item and follow the instructions for :ref:`using CGDISK + to set up your Clear Linux hard drive`. + +.. _choose-target-device: + +Choose target device for installation +===================================== + +The :guilabel:`Choose target device for installation` menu, shown in figure 3, +displays the current device and partition information. In +this example, ``/dev/sda`` is the only drive on the system with three +partitions defined and assigned. The menu cursor highlights the partition +to install |CL| onto. + +#. Select the :guilabel:`< Partition /dev/sda >` button and press the + :kbd:`Enter` key. + + .. note:: + + To return to the previous menu, press the :kbd:`Tab` key + to highlight the :guilabel:`< Previous >` button and press :kbd:`Enter` + to return to the previous menu. + + .. figure:: figures/bare-metal-manual-install-3.png + :scale: 50 % + :alt: Choose target device for installation + + Figure 3: :guilabel:`Choose target device for installation` + +#. After selecting the :guilabel:`< Partition /dev/sda >` button you will be + presented with a warning screen as shown in figure 4 with the + :guilabel:`< No >` button highlighted. If you are satisfied this is the + device that you want to install |CL| onto, use the :kbd:`Tab` key to + highlight the :guilabel:`< Yes >` button and press :kbd:`Enter` to proceed + to the next step. + + .. figure:: figures/bare-metal-manual-install-4.png + :scale: 50 % + :alt: Device installation warning + + Figure 4: :guilabel:`Device installation warning` + +Additional manual installer settings +==================================== + +The next steps of the manual installer setup process allows you to: + +* Set up any additional command-line parameters to the kernel. +* Create a hostname for your system. +* Set up an administrative user. +* Install additional software bundles. +* Optionally create a static IP address for your system. + +#. The :guilabel:`Append to kernel cmdline` menu shown in figure 5 allows you + to add additional options to the kernel command-line to further customize + your installation. Normally this is not required but if there are + specific options that need to be set in the Linux kernel you can enter them + here. For a complete list of command-line parameters for the Linux kernel + you can visit the official documentation of the `latest kernel's + command-line parameters`_. + + Add any additional command-line parameters in the :guilabel:`Append to + cmdline:` field and then press the :kbd:`Tab` key to move to the + :guilabel:`< Next >` button and press :kbd:`Enter`. + + .. figure:: figures/bare-metal-manual-install-5.png + :scale: 50 % + :alt: Append to kernel cmdline + + Figure 5: :guilabel:`Append to kernel cmdline` + +#. The :guilabel:`Configuring Hostname` menu shown in figure 6 suggests + a hostname string prefaced with "clr". You can make your hostname anything + meaningful and unique. Once complete, press the :kbd:`Tab` key to move to + the :guilabel:`< Next >` button and press :kbd:`Enter`. + + .. figure:: figures/bare-metal-manual-install-6.png + :scale: 50 % + :alt: Configure Hostname + + Figure 6: :guilabel:`Configure Hostname` + + To learn more about hostname naming conventions and restrictions, you can + check this `wiki page about hostnames`_. + +#. The :guilabel:`User configuration` menu shown in figure 7 allows you to + create a userid with administrative privileges. If you do not want to create a + user at this time, select :guilabel:`< No user creation (login as root) >` to + skip this step and proceed to the :guilabel:`Bundle selector` menu. + + .. figure:: figures/bare-metal-manual-install-7.png + :scale: 50 % + :alt: User configuration + + Figure 7: :guilabel:`User configuration` + + #. To create a userid, select the :guilabel:`< Create an administrative user >` + field and press :kbd:`Enter` to go to the the next screen. + #. You will be presented with the second :guilabel:`User configuration` menu + shown in figure 8. You are only required to fill in the + :guilabel:`Username`, :guilabel:`Password`, and :guilabel:`Confirm password` + fields. + #. To give the user administrative privileges, select the + :guilabel:`Add user to the sudoers?` field to add the user to the ``wheel`` + group. This enables using the :command:`sudo` command for the new user. + + Once you have entered all the data for this menu, press the :kbd:`Tab` key + to highlight the :guilabel:`< Next >` button and press :kbd:`Enter`. + + .. figure:: figures/bare-metal-manual-install-8.png + :scale: 50 % + :alt: User configuration - create user + + Figure 8: :guilabel:`User configuration - create user` + +#. The :guilabel:`Bundle selector` menu shown in figure 9 allows you to + add additional software bundles to your initial |CL| installation. In this + example we select all additional bundles offered by pressing the + :kbd:`Tab` key to go to each field and pressing the :kbd:`Spacebar` or the + :kbd:`Enter` key to select each bundle. This menu will also list the + additional :guilabel:` --- required ---` software bundles that will be + installed during the |CL| installation. + + .. figure:: figures/bare-metal-manual-install-9.png + :scale: 50 % + :alt: Bundle selector + + Figure 9: :guilabel:`Bundle selector` + +For a complete description of the content of these additional bundles, go to +the :ref:`software bundle list` and select the name for a +specific bundle to show the contents within the bundle. + +Target system network configuration +=================================== + +Before the actual |CL| installation begins, we must set up the +:guilabel:`Network configuration` shown in figure 10. Select +:guilabel:`< Use DHCP >` to have an IP address automatically assigned when +the system boots up. + +.. figure:: figures/bare-metal-manual-install-10.png + :scale: 50 % + :alt: Network configuration + + Figure 10: :guilabel:`Network configuration` + +You can optionally set up a static IP address for your system. Select the +:guilabel:`< Use static IP configuration >` menu item and fill in the +required fields. + +Begin installation +****************** + +The |CL| installer is now ready to use the information you provided on the +previous sections to begin the actual installation. Figure 11 shows the +installer's prompt to begin the installation. + +.. figure:: figures/bare-metal-manual-install-11.png + :scale: 50 % + :alt: Begin installation + + Figure 11: :guilabel:`Begin installation` + +When you are satisfied with the information entered, select the +:guilabel:`< Yes >` button and press :kbd:`Enter` to begin installing |CL|. + +The |CL| Installation begins and each step shows its status as it progresses +through the manual installation process as shown in figure 12. + +.. figure:: figures/bare-metal-manual-install-12.png + :scale: 50 % + :alt: Installation status + + Figure 12: :guilabel:`Installation status` + +Once all steps have completed, you will see the ``Successful installation`` +status message and the :guilabel:`< Ok >` button is highlighted as shown in +figure 13. Press the :kbd:`Enter` key to continue. + +.. figure:: figures/bare-metal-manual-install-13.png + :scale: 50 % + :alt: Installation complete + + Figure 13: :guilabel:`Installation complete` + +Congratulations! The installation completed successfully and the system will +reboot. Figure 14 shows the installer's final prompt. Press the :kbd:`Enter` +key and remove the USB media while the system restarts. + +.. figure:: figures/bare-metal-manual-install-14.png + :scale: 50 % + :alt: Successful Installation + + Figure 14: :guilabel:`Successful Installation` + +Once the system boots, the Gnome Desktop sign-in screen shown in figure 15 +appears: + + .. figure:: figures/bare-metal-manual-install-15.png + :scale: 50 % + :alt: Gnome sign-in screen + + Figure 15: :guilabel:`Gnome sign-in screen` + +Click on the :guilabel:`username` you created, enter the password, and you +will be logged into the system. The Gnome 3 desktop appears as shown in +figure 16: + +.. figure:: figures/bare-metal-manual-install-16.png + :scale: 50 % + :alt: Gnome Desktop Manager + + Figure 16: :guilabel:`Gnome Desktop Manager` + +**Congratulations!** + +You have successfully installed |CL| on a bare metal system using the +manual installation method. + +Visit our :ref:`tutorials ` section for examples on using your +|CL| system. + +.. _`information about stateless`: + https://clearLinux.org/features/stateless + +.. _`wiki page about Hostnames`: + https://en.wikipedia.org/wiki/Hostname + +.. _`learn more about telemetry.`: + https://clearLinux.org/features/telemetry + +.. _`latest kernel's command-line parameters`: + https://www.kernel.org/doc/html/latest/admin-guide/kernel-parameters.html diff --git a/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst b/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst new file mode 100755 index 00000000..abdca674 --- /dev/null +++ b/source/clear-linux/get-started/bare-metal-install/cgdisk-manual-install.rst @@ -0,0 +1,304 @@ +.. _cgdisk-manual-install: + +Clear Linux partitioning using CGDISK +##################################### + +These instructions guide you through the initial setup of your hard drive +partitions using the :command:`cgdisk` utility as part of the |CL| manual +installation process. If you do not wish to continue creating your own +partitions, :ref:`return to the bare metal manual installation +`. + +Prerequisites +************* + +This guide assumes that you have followed the :ref:`bare-metal-install` +guide and intend to :ref:`install Clear Linux manually +` by choosing the :guilabel:`< Manual(Advanced) >` +menu item on the :guilabel:`Choose Installation Type` |CL| Installer setup +menu as shown in figure 1: + +.. figure:: figures/cgdisk-manual-install-1.png + :scale: 50 % + :alt: Choose installation Type + + Figure 1: :guilabel:`Choose installation Type` + +Partition using CGDISK +********************** + +We use the :command:`cgdisk` application to create a +:abbr:`GPT (GUID Partition Table)` since |CL| only supports the +:abbr:`UEFI (Unified Extensible Firmware Interface)` specification. For a +complete description of the :command:`cgdisk` utility and how to use it, visit +Rod Smith's website for a `GPT fdisk tutorial`_. + +In this guide, we intend to use an unpartitioned hard drive for the |CL| +installation. + +#. On the :guilabel:`Choose partitioning method` menu, shown in figure 2, + select the :guilabel:`< Manually configure mounts and partitions >` menu + item to manually partition your hard drive. + + .. figure:: figures/cgdisk-manual-install-2.png + :scale: 50 % + :alt: Choose partitioning method + + Figure 2: :guilabel:`Choose partitioning method` + + The screen then shows the current device on your system you can partition. + In this example, shown in figure 3, :file:`/dev/sda` is available but + does not have any partitions defined. + +#. Select the :guilabel:`< Partition /dev/sda >` menu item and press + :kbd:`Enter` to begin the process of modifying this disk. + + .. figure:: figures/cgdisk-manual-install-3.png + :scale: 50 % + :alt: Choose a drive to partition using cgdisk tool + + Figure 3: :guilabel:`Choose a drive to partition using cgdisk tool` + + The :command:`cgdisk` application starts and displays the settings for + :file:`/dev/sda` as shown in figure 4. + + .. figure:: figures/cgdisk-manual-install-4.png + :scale: 50 % + :alt: cgdisk + + Figure 4: :guilabel:`cgdisk` + +Linux Partition setup +********************* + +In order to properly set up the |CL| partitioning scheme, we create three +partitions using the :command:`cgdisk` utility in the following order: + + #. EFI boot partition + #. Linux swap partition + #. Linux root partition + +For a complete understanding of these partitions, you can review the +`Linux partitioning scheme`_ information. + +Create the EFI boot partition +============================= + +#. With the free space highlighted in the :command:`cgdisk` utility, + you can either select the :guilabel:`[ New ]` button and press :kbd:`Enter` + or press the :kbd:`N` key to define a new partition. + + The utility prompts you to enter the first sector. Press the :kbd:`Enter` + key to accept the default value shown. + + .. note:: + In this example, the first sector starts at 2048. For more information + about alignment using the cgdisk tool, see + `Rod Smith's Partitioning Advice about alignment`_. + +#. The program then prompts you for the size of the partition. To create a + 512MB partition, enter 512M and press :kbd:`Enter` as shown in figure 5: + + .. figure:: figures/cgdisk-manual-install-5.png + :scale: 50 % + :alt: cgdisk - New + + Figure 5: :guilabel:`cgdisk - New partition` + +#. To define the type of partition, the :command:`cgdisk` utility has + pre-defined partition types. Press the :kbd:`L` key to show the hex codes + you can use. Use these codes to set the correct + :abbr:`GUID (Globally unique identifier)` for *GPT partition types* as + shown in figure 6: + + .. figure:: figures/cgdisk-manual-install-6.png + :scale: 50 % + :alt: cgdisk - hex codes for partition types + + Figure 6: :guilabel:`cgdisk - hex codes for partition types` + + We need to use the following three codes for our partitions: + + * ef00 - EFI System + * 8200 - Linux swap + * 8300 - Linux filesystem + +#. To create the EFI boot partition, enter ``ef00`` as the hexcode for this + partition and press :kbd:`Enter`. + +#. To name the partition, enter ``boot`` and press :kbd:`Enter` to finish + setting up the EFI boot partition. The utility shows the first partition as + an ``EFI System`` 512MiB partition named ``boot`` as shown in figure 7: + + .. figure:: figures/cgdisk-manual-install-7.png + :scale: 50 % + :alt: cgdisk - boot partition defined + + Figure 7: :guilabel:`cgdisk - boot partition defined` + +Create the Linux swap partition +=============================== + +Next, we must create the Linux swap partition. In figure 7, notice the two +areas defined as free space. We created the first 1007.0 KiB free space area +when we started the EFI boot partition at sector 2048. For more information +about it, review `Rod Smith's Partitioning advice about alignment`_. + +#. Move your cursor to highlight the larger free space of 334.8 GiB at the + bottom of the partition list before you begin to create the Linux swap + partition as shown in figure 8: + + .. figure:: figures/cgdisk-manual-install-8.png + :scale: 50 % + :alt: cgdisk - free space selection + + Figure 8: :guilabel:`cgdisk - free space selection` + +#. To create the Linux swap partition, with the largest free space + highlighted, select the :guilabel:`[ New ]` button or press the :kbd:`N` + key and enter the following values for the Linux swap partition: + + .. code-block:: console + + First sector: press :kbd:`Enter` to select the default value + Size in sectors: 4G + Hex code or GUID: 8200 + Enter new partition name: swap + + Your :command:`cgdisk` partition list should now look like figure 9. + + .. figure:: figures/cgdisk-manual-install-9.png + :scale: 50 % + :alt: cgdisk - swap partition defined + + Figure 9: :guilabel:`cgdisk - swap partition defined` + +Create the Linux filesystem partition +************************************* + +Lastly, we must create the the Linux filesystem partition to use it as the +root mount point for you |CL| installation. + +#. Highlight the largest free space entry at the bottom of the list and select + the :guilabel:`[ New ]` button or press the :kbd:`N` key and enter the + following values to create the Linux filesystem partition: + + .. code-block:: console + + First sector: press :kbd:`Enter` to select the default value + Size in sectors: press :kbd:`Enter` to select the default value, which + is the remainder of available space on the disk + Hex code or GUID: 8300 + Enter new partition name: root + + With all the partitions now defined, you should see a list similar to what + is shown in figure 10: + + .. figure:: figures/cgdisk-manual-install-10.png + :scale: 50 % + :alt: cgdisk - defined partitions + + Figure 10: :guilabel:`cgdisk - defined partitions` + +#. If you are satisfied that the partition scheme is correct, you need to + write this GPT to the hard drive. Select the :guilabel:`[ Write ]` button + or press the :kbd:`W` key and the :command:`cgdisk` program prompts with: + + .. code-block:: console + + Are you sure you want to write the partition table to disk? (yes or no) + +#. Enter ``yes`` and press :kbd:`Enter` to write this data to the hard drive + and then select the :guilabel:`[ Quit ]` button or press :kbd:`Q` to exit + the :command:`cgdisk` utility. + +#. You see the partitions that were created as shown in figure 11. Move your + cursor to the :guilabel:`< Next >` button and press :kbd:`Enter`. + + .. figure:: figures/cgdisk-manual-install-11.png + :scale: 50 % + :alt: defined partitions + + Figure 11: :guilabel:`defined partitions` + +Set the mount points +******************** + +The :guilabel:`Set mount points` menu sets the mount points that the |CL| +installer uses for your |CL| installation and is shown in figure 12. + +.. figure:: figures/cgdisk-manual-install-12.png + :scale: 50 % + :alt: Set mount points + + Figure 12: :guilabel:`Set mount points` + +In this menu you need to set the mount points for the boot and root partitions +and select to format them. + +#. Highlight the EFI System partition type menu entry and press the + :kbd:`Enter` key to edit this item. The :guilabel:`Set mount point of + sda1` menu is be shown. + + #. For the :guilabel:`Enter mount point:` type `/boot` and press + :kbd:`Enter`. + + #. Enable formatting the partition by checking the :guilabel:`[ ] Format` + toggle field. + + Figure 13 shows the entered information. Select the :guilabel:`< Yes >` + button and press :kbd:`Enter`. + + .. figure:: figures/cgdisk-manual-install-13.png + :scale: 50 % + :alt: Set mount point of sda1 + + Figure 13: :guilabel:`Set mount point of sda1` + +#. Do the same for the Linux filesystem partition by highlighting the + :guilabel:`Linux filesystem` menu entry and entering the information shown + in figure 14 to set the :guilabel:`Enter mount point:` to :file:`/` and + enable formatting: + + .. figure:: figures/cgdisk-manual-install-14.png + :scale: 50 % + :alt: Set mount point of sda3 + + Figure 14: :guilabel:`Set mount point of sda3` + + The final :guilabel:`Set mount points` menu item looks like figure 15: + + .. figure:: figures/cgdisk-manual-install-15.png + :scale: 50 % + :alt: Set mount point completed + + Figure 15: :guilabel:`Set mount points completed` + +#. Select the :guilabel:`< Next >` button and press :kbd:`Enter` to proceed to + the :guilabel:`Warning!` menu to accept your changes as shown in figure 16. + + .. figure:: figures/cgdisk-manual-install-16.png + :scale: 50 % + :alt: Warning + + Figure 16: :guilabel:`Warning` + + Highlight the :guilabel:`< Yes >` button and press :kbd:`Enter` to accept + these changes and move on to the next step of the |CL| manual install + process. + + This completes the process of manually setting up your hard drive + partitions and you can now :ref:`continue with the Clear Linux manual + install`. + +.. _`GPT fdisk tutorial`: + http://www.rodsbooks.com/gdisk/ + +.. _`Rod Smith's Partitioning Advice about alignment`: + http://www.rodsbooks.com/gdisk/advice.html#alignment + +.. _`information about swupd`: + https://clearlinux.org/features/software-update + +.. _`Linux partitioning scheme`: + https://wiki.archlinux.org/index.php/partitioning#Partition_scheme \ No newline at end of file diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-1.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-1.png new file mode 100644 index 00000000..957bf79a Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-1.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-10.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-10.png new file mode 100644 index 00000000..b0cb9b0b Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-10.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-11.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-11.png new file mode 100644 index 00000000..b0cc483f Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-11.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-12.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-12.png new file mode 100644 index 00000000..76c5fe87 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-12.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-2.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-2.png new file mode 100644 index 00000000..b82f34fd Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-2.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-3.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-3.png new file mode 100644 index 00000000..4f6c3bb3 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-3.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-4.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-4.png new file mode 100644 index 00000000..1d4c4a50 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-4.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-5.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-5.png new file mode 100644 index 00000000..217690c2 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-5.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-6.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-6.png new file mode 100644 index 00000000..9a7024c7 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-6.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-7.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-7.png new file mode 100644 index 00000000..9515eb20 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-7.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-8.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-8.png new file mode 100644 index 00000000..7d366096 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-8.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-9.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-9.png new file mode 100644 index 00000000..35911d99 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-install-9.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-1.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-1.png new file mode 100644 index 00000000..cb023cf5 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-1.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-10.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-10.png new file mode 100644 index 00000000..d8a3f5f3 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-10.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-11.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-11.png new file mode 100644 index 00000000..107febc4 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-11.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-12.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-12.png new file mode 100644 index 00000000..66751565 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-12.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-13.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-13.png new file mode 100644 index 00000000..9a14d9e1 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-13.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-14.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-14.png new file mode 100644 index 00000000..50fdc0ed Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-14.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-15.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-15.png new file mode 100644 index 00000000..7b538705 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-15.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-16.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-16.png new file mode 100644 index 00000000..df61163b Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-16.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-2.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-2.png new file mode 100644 index 00000000..b52d3f3b Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-2.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-3.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-3.png new file mode 100644 index 00000000..2c5ff918 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-3.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-4.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-4.png new file mode 100644 index 00000000..a5897712 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-4.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-5.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-5.png new file mode 100644 index 00000000..54acbe3e Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-5.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-6.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-6.png new file mode 100644 index 00000000..0d2fe7f2 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-6.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-7.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-7.png new file mode 100644 index 00000000..a98d99b8 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-7.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-8.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-8.png new file mode 100644 index 00000000..8401cf67 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-8.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-9.png b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-9.png new file mode 100644 index 00000000..ed43c853 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/bare-metal-manual-install-9.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-1.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-1.png new file mode 100644 index 00000000..cb023cf5 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-1.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-10.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-10.png new file mode 100644 index 00000000..8d977f83 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-10.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-11.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-11.png new file mode 100644 index 00000000..91a2d13d Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-11.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-12.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-12.png new file mode 100644 index 00000000..542c3101 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-12.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-13.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-13.png new file mode 100644 index 00000000..7393ad38 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-13.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-14.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-14.png new file mode 100644 index 00000000..a558b8ec Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-14.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-15.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-15.png new file mode 100644 index 00000000..cc885b6d Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-15.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-16.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-16.png new file mode 100644 index 00000000..a5897712 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-16.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-2.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-2.png new file mode 100644 index 00000000..4120d770 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-2.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-3.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-3.png new file mode 100644 index 00000000..97303025 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-3.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-4.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-4.png new file mode 100644 index 00000000..d81b6d95 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-4.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-5.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-5.png new file mode 100644 index 00000000..75163ffa Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-5.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-6.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-6.png new file mode 100644 index 00000000..1528b667 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-6.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-7.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-7.png new file mode 100644 index 00000000..f9b007fc Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-7.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-8.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-8.png new file mode 100644 index 00000000..7c41a266 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-8.png differ diff --git a/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-9.png b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-9.png new file mode 100644 index 00000000..b5b351c3 Binary files /dev/null and b/source/clear-linux/get-started/bare-metal-install/figures/cgdisk-manual-install-9.png differ diff --git a/source/clear-linux/get-started/get-started.rst b/source/clear-linux/get-started/get-started.rst index ad5149e0..f08f435f 100644 --- a/source/clear-linux/get-started/get-started.rst +++ b/source/clear-linux/get-started/get-started.rst @@ -16,6 +16,8 @@ information and instructions on how to complete pre-install tasks. ../reference/system-requirements bare-metal-install/bare-metal-install + bare-metal-install/bare-metal-manual-install + bare-metal-install/cgdisk-manual-install virtual-machine-install/virtual-machine-install live-image compatibility-check diff --git a/source/clear-linux/guides/maintenance/enable-user-space.rst b/source/clear-linux/guides/maintenance/enable-user-space.rst new file mode 100644 index 00000000..a21fab96 --- /dev/null +++ b/source/clear-linux/guides/maintenance/enable-user-space.rst @@ -0,0 +1,180 @@ +.. _enable-user-space: + +Create and enable a new user space +################################## + +This section provides steps to complete the following basic setup tasks for +a newly installed |CL| system: + +* Create a new user +* Update the OS to its most current version using `swupd`. +* Install the most common applications for system administrators and + developers using bundles. +* Setup a new user. +* Setup `sudo` privileges for that new user. +* Install a GUI using those `sudo` privileges. + +.. note:: + You must be logged in as the root user to complete the tasks in this + section. + +Create a new user +================= + +To create a new user and set a password for that user, enter the following +commands as a root user: + +.. code-block:: console + + useradd + passwd + +Replace with the name of the user account you want to create and +with the password for said user. The :command:`passwd` command prompts you +to enter a new password and then retype the new password for the new user +account you just created. + +Install and update software +=========================== + +|CL| has a unique application and architecture to add and update applications +and to perform system updates called software update utility or +:command:`swupd`. Software applications are installed as bundles using the +sub-command :command:`bundle-add`. + +The `os-clr-on-clr` bundle installs the vast majority of +applications useful to a system administrator or a developer. The bundle +contains other bundles such as `sysadmin-basic`, `editors`, `c-basic`, +`dev-utils-dev`, and other useful packages. + +Install the `os-clr-on-clr` bundle using the software update +utility: + +.. code-block:: console + + swupd bundle-add os-clr-on-clr + +.. note:: + + The image we installed may not be the latest version of |CL| available on + the server. However, whenever the command + :command:`swupd bundle-add ` runs, the OS is updated to the latest + available version. Our website provides more `information about swupd`_. + +We provide the full list of bundles and packages installed with the +`os-clr-on-clr`_ bundle. Additionally, we have listed +`all Clear Linux bundles`_, active or deprecated. Click any bundle on the +list to view the manifest of the bundle. + +Finish setting up your new user +=============================== + +Before logging off as root and logging into your new user account, we must +enable the :command:`sudo` command for your new ``. + +To be able to execute all applications with root privileges, we must add the +`` to the `wheel group`_ and enable the wheel group in the +:file:`/etc/sudoers` file. + +#. Add `` to the wheel group: + + .. code-block:: console + + usermod -G wheel -a + +#. Open the :file:`/etc/sudoers` file: + + .. code-block:: console + + vi /etc/sudoers + + .. note:: + + Normally, we would use the visudo script to edit the :file:`/etc/sudoers` + file to safely modify the contents of the file. In this instance, the + file does not exist yet. Therefore, we create the initial instance of + the file. + +#. In the vi\* editor window, press the :kbd:`o` key to open a new line. + +#. Add the following line to the file: + + .. code-block:: console + + %wheel ALL=(ALL) ALL + +#. To save the changes to the file and exit vi, press the :kbd:`ESC` key + followed by the :kbd:`:` and :kbd:`x` keys. + + .. important:: + + Creating the file logged as the root user keeps the permissions of the + file with the root user. + +#. Now, we can log out of root and into our new ``. + + To log off as root, enter :command:`exit`. + + The command will bring you back to the `login:` prompt. + +#. Enter the new `` and the password you created earlier. + + You will now be in the home directory of ``. The bundle + `os-clr-on-clr`_ contains the majority of applications that a developer or + system administrator would want but it does not include a graphical user + interface. The `desktop` bundle includes the Gnome Desktop Manager and + additional supporting applications. + +Install a GUI to test sudo +-------------------------- + +To test the :command:`sudo` command and ensure it is set up correctly, +install the Gnome Desktop Manager (gdm) and start it. + +#. To install Gnome using :command:`swupd`, enter the following command: + + .. code-block:: console + + sudo swupd bundle-add desktop + +#. To start the Gnome Desktop Manager, enter the following command: + + .. code-block:: console + + systemctl start gdm + +#. The system prompts you to authenticate the user. Enter the password for + `` and the Gnome Desktop should start as shown in figure 13: + + .. figure:: figures/gnomedt.png + :scale: 50 % + :alt: Gnome Desktop + + Figure 13: :guilabel:`Gnome Desktop` + +#. To start the Gnome Desktop each time you start your system, enter + the following command: + + .. code-block:: console + + systemctl enable gdm + +Next steps +========== + +With your system now running |CL| many paths are open for you. + +Visit our :ref:`tutorials ` page for examples on using your |CL| +system. + +.. _`information about swupd`: + https://clearlinux.org/features/software-update + +.. _`os-clr-on-clr`: + https://github.com/clearlinux/clr-bundles/blob/master/bundles/os-clr-on-clr + +.. _`all Clear Linux bundles`: + https://github.com/clearlinux/clr-bundles/tree/master/bundles + +.. _`wheel group`: + https://en.wikipedia.org/wiki/Wheel_(Unix_term) diff --git a/source/clear-linux/get-started/bare-metal-install/figures/gnomedt.png b/source/clear-linux/guides/maintenance/figures/gnomedt.png old mode 100644 new mode 100755 similarity index 100% rename from source/clear-linux/get-started/bare-metal-install/figures/gnomedt.png rename to source/clear-linux/guides/maintenance/figures/gnomedt.png diff --git a/source/clear-linux/guides/maintenance/maintenance.rst b/source/clear-linux/guides/maintenance/maintenance.rst index 8d4f0c4b..c7806bcc 100644 --- a/source/clear-linux/guides/maintenance/maintenance.rst +++ b/source/clear-linux/guides/maintenance/maintenance.rst @@ -9,7 +9,9 @@ maintaining |CLOSIA| after :ref:`installation ` is completed. .. toctree:: :maxdepth: 2 + enable-user-space update bulk-provision mixer validate-signatures + telemetry-enable diff --git a/source/clear-linux/guides/maintenance/telemetry-enable.rst b/source/clear-linux/guides/maintenance/telemetry-enable.rst new file mode 100644 index 00000000..80a33839 --- /dev/null +++ b/source/clear-linux/guides/maintenance/telemetry-enable.rst @@ -0,0 +1,112 @@ +.. _telemetry-enable: + +Enable and disable telemetry in Clear Linux +########################################### + +|CLOSIA| includes a telemetry solution as part of the OS that records events +of interest and reports them back to the development team via the telemetrics +daemon, :command:`telemd`. This functionality is maintained in the +``telemetrics`` software bundle. + +.. note:: + The telemetry functionality adheres to `Intel's privacy policies`_ + regarding the collection and use of :abbr:`PII (Personally Identifiable + Information)` and is open source. Specifically, no intentionally + identifiable information about the user or system owner is collected. + +End users may enable or disable the telemetry component of |CL| or even +redirect where the records go if they wish to collect records for themselves. + +Install the telemetry software bundle +************************************* + +During the initial installation of |CL| you are requested to join the +stability enhancement program and allow |CLOSIA| to collect anonymous reports +to improve system stability. If you chose not to join this program at that +time then the telemetry software bundle is not added to your system. + +To install the telemetry bundle, enter the following command as either the +root user or with :command:`sudo` priveleges: + +.. code-block:: console + + sudo swupd bundle-add telemetrics + +This adds the telemetrics-client to your system and you will automatically +opt-in for the service. + +Enable Telemetry +**************** + +To start telemetry on your system run the following command: + +.. code-block:: console + + sudo telemctl start + +This enables and starts the :command:`telemd` daemon and your system will +begin to send telemetry data to the server defined in the file +:file:`/etc/telemetrics/telemetrics.conf`. If this file does not exist, the +:command:`telemd` daemon will use the file +:file:`/usr/share/defaults/telemetrics/telemetrics.conf`. + +Disable Telemetry +***************** + +To disable the telemetry daemon run the following command: + +.. code-block:: console + + sudo telemctl stop + +Opt-out of telemetry +******************** + +To stop sending telemetrics data from your system, opt out of the +telemetry service: + +.. code-block:: console + + sudo telemctl opt-out + +This creates the file :file:`/etc/telemetrics/opt-out` and stops the +telemetry services. + +Opt-in to telemetry +******************* + +Conversely, to opt-in to the telemetry services, simply enter the opt-in +command and start the service: + +.. code-block:: console + + sudo telemctl opt-in + +This removes the file :file:`/etc/telemetrics/opt-out` file, if it exists, +and starts the telemetry services. + +.. note:: + To opt-in but not immediately start telemetry services, you will need to + run the command :command:`sudo telemctl stop` after the :command:`opt-in` + command is entered. Once you are ready to start the service, enter the + command :command:`sudo telemctl start`. + +Remove the telemetry software bundle +************************************ + +To completely remove telemetrics from your system, use the command +:command:`swupd` to remove the telemetry software bundle: + +.. code-block:: console + + sudo swupd bundle-remove telemetrics + +Additional resources +******************** + +https://clearlinux.org/features/telemetry + +https://github.com/clearlinux/telemetrics-client + +.. _`Intel's privacy policies`: + http://www.intel.com/content/www/us/en/privacy/intel-privacy.html diff --git a/source/clear-linux/reference/bundles/bundles.html b/source/clear-linux/reference/bundles/bundles.html index 9284dd53..b7aa3a78 100644 --- a/source/clear-linux/reference/bundles/bundles.html +++ b/source/clear-linux/reference/bundles/bundles.html @@ -405,7 +405,7 @@ - kernel-hyperv + kernel-hyperv-lts Active

Run the hyperv specific LTS kernel diff --git a/source/clear-linux/tutorials/hadoop.rst b/source/clear-linux/tutorials/hadoop.rst new file mode 100644 index 00000000..a489c855 --- /dev/null +++ b/source/clear-linux/tutorials/hadoop.rst @@ -0,0 +1,256 @@ +.. _hadoop: + +Set up a single node cluster with Hadoop\* +########################################## + +This tutorial walks you through the process of installing, configuring, and +running Apache\* Hadoop on |CLOSIA|. The Apache Hadoop software library is a +framework for distributed processing of large data sets across clusters of +computers using simple programming models. It is designed to scale up from +single servers to thousands of machines, with each machine offering local +computation and storage. + +Prerequisites +************* + +Before following this tutorial, you should follow the +:ref:`bare-metal-install` to ensure you have installed |CLOSIA|. + +Before you install any new packages, update |CL| with the following command: + +.. code-block:: bash + + sudo swupd update + +For the purposes of this tutorial, we will install Hadoop in a single machine +running both the master and slave daemons. + +Install Apache Hadoop +********************* + +Apache Hadoop is included in the `big-data-basic` bundle. To install the +framework, enter the following command: + +.. code-block:: bash + + sudo swupd bundle-add big-data-basic + +Configure Apache Hadoop +*********************** + +#. To create the configuration directory, enter the following command: + + .. code-block:: bash + + sudo mkdir /etc/hadoop + +#. Copy the defaults from :file:`/usr/share/defaults/hadoop` to + :file:`/etc/hadoop` with the following command: + + .. code-block:: bash + + $ sudo cp /usr/share/defaults/hadoop/* /etc/hadoop + + .. note:: Since |CL| is a stateless system, you should never modify the + files under the :file:`/usr/share/defaults` directory. The software + updater will overwrite those files. + +Once all the configuration files are in :file:`/etc/hadoop`, we must edit +them to fit our needs. The `NameNode` server is the master server. It manages +the namespace of the files system and regulates the clients' access to files. +The first file we edit, :file:`/etc/hadoop/core-site.xml`, informs the Hadoop +daemon where `NameNode` is running. + +In this tutorial, our `NameNode` runs in our `localhost`. Follow these steps +to set it up correctly: + +#. Open the :file:`/etc/hadoop/core-site.xml` file using the editor of your + choice and modify the file as follows: + + .. code-block:: xml + + + + + + fs.default.name + hdfs://localhost:9000 + + + +#. Edit the :file:`/etc/hadoop/hdfs-site.xml` file. This file configures the + :abbr:`HDFS (Hadoop Distributed File System)` daemons. This configuration + includes the list of permitted and excluded data nodes and the size of + said blocks. In this example, we are setting the number of block + replication to 1 from the default of 3 as follows: + + .. code-block:: xml + :emphasize-lines: 6 + + + + + + dfs.replication + 1 + + + dfs.permission + false + + + +#. Edit the :file:`/etc/hadoop/mapred-site.xml` file. This file configures + all daemons related to `MapReduce`: `JobTracker` and `TaskTrackers`. With + `MapReduce`, Hadoop can process big amounts of data in multiple systems. In + our example, we set :abbr:`YARN (Yet Another Resource Manager)` as our + runtime framework for executing `MapReduce` jobs as follows: + + .. code-block:: xml + :emphasize-lines: 5,6 + + + + + + mapreduce.framework.name + yarn + + + +#. Edit the :file:`/etc/hadoop/yarn-site.xml` file. This file configures all + daemons related to `YARN`: `ResourceManager` and `NodeManager`. In our + example, we implement the `mapreduce_shuffle` service, which is the + default as follows: + + .. code-block:: xml + :emphasize-lines: 4,5,8,9 + + + + + yarn.nodemanager.aux-services + mapreduce_shuffle + + + yarn.nodemanager.auxservices.mapreduce.shuffle.class + org.apache.hadoop.mapred.ShuffleHandler + + + +Configure your SSH key +********************** + +#. Create a SSH key. If you already have one, skip this step. + + .. code-block:: bash + + sudo ssh-keygen -t rsa + + +#. Copy the key to your authorized keys. + + .. code-block:: bash + + sudo cat /root/.ssh/id_rsa.pub | sudo tee -a /root/.ssh/authorized_keys + +#. Log into the localhost. If no password prompt appears, you are ready to + run the Hadoop daemons. + + .. code-block:: bash + + sudo ssh localhost + +Run the Hadoop daemons +********************** + +With all the configuration files properly edited, we are ready to start the +daemons. + +When we format the `NameNode` server, it formats the meta-data related to +data nodes. Thus, all the information on the data nodes is lost and the nodes +can be reused for new data. + +#. Format the `NameNode` server with the following command: + + .. code-block:: bash + + sudo hdfs namenode -format + +#. Start the DFS in `NameNode` and `DataNodes` with the following command: + + .. code-block:: bash + + sudo start-dfs.sh + +#. The console output should be similar to: + + .. code-block:: console + + Starting namenodes on [localhost] + The authenticity of host 'localhost (::1)' can't be established. + ECDSA key fingerprint is + SHA256:97e+7TnomsS9W7GjFPjzY75HGBp+f1y6sA+ZFcOPIPU. + Are you sure you want to continue connecting (yes/no)? + + Enter `yes` to continue. + +#. Start the `YARN` daemons `ResourceManager` and `NodeManager` with the + following command: + + .. code-block:: bash + + sudo start-yarn.sh + +#. Ensure everything is running as expected with the following command: + + .. code-block:: bash + + sudo jps + +#. The console output should be similar to: + + .. code-block:: console + + 22674 DataNode + 26228 Jps + 22533 NameNode + 23046 ResourceManager + 22854 SecondaryNameNode + 23150 NodeManager + +Run the MapReduce wordcount example +*********************************** + +#. Create the input directory. + + .. code-block:: bash + + sudo hdfs dfs -mkdir -p /user/root/input + +#. Copy a file from the local file system to the HDFS. + + .. code-block:: bash + + sudo hdfs dfs -copyFromLocal local-file /user/root/input + +#. Run the `wordcount` example. + + .. code-block:: bash + + sudo hadoop jar /usr/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.8.0.jar wordcount input output + +#. Read output file "part-r-00000". This file contains the number of times + each word appears in the file. + + .. code-block:: bash + + sudo hdfs dfs -cat /user/root/output/part-r-00000 + +**Congratulations!** + +You successfully installed and setup a single node Hadoop cluster. +Additionally, you ran a simple wordcount example. + +Your single node Hadoop cluster is up and running! + diff --git a/source/clear-linux/tutorials/tutorials.rst b/source/clear-linux/tutorials/tutorials.rst index 04998c58..8a5a646e 100644 --- a/source/clear-linux/tutorials/tutorials.rst +++ b/source/clear-linux/tutorials/tutorials.rst @@ -15,4 +15,4 @@ specific |CLOSIA| use cases. azure multi-boot/multi-boot hadoop - fmv + fmv \ No newline at end of file diff --git a/source/index.rst b/source/index.rst index 4841c961..8b4c503e 100644 --- a/source/index.rst +++ b/source/index.rst @@ -8,7 +8,6 @@ Clear technologies - Documentation clear-linux/clear-linux clear-containers/clear-containers - ciao/ciao License and disclaimers =======================