From c2147039bb0c4d00ae3bc1f0a999991c5577f012 Mon Sep 17 00:00:00 2001 From: Alberto Murillo Date: Tue, 20 Sep 2016 12:13:17 -0500 Subject: [PATCH 1/6] ciao-deploy.rst: Update passwords Update passwords shown un ciao-deploy to match those in https://github.com/clearlinux/clear-config-management/tree/master/examples/ciao Signed-off-by: Alberto Murillo --- source/ciao-deploy.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/ciao-deploy.rst b/source/ciao-deploy.rst index 6cf68042..06d04292 100644 --- a/source/ciao-deploy.rst +++ b/source/ciao-deploy.rst @@ -121,7 +121,7 @@ was specified in the ``groups_var/all`` file: .. code-block:: console - # ciao-cli -identity=https://ciao-controller.example.com:35357 -username admin -password secret -controller=ciao-controller.example.com node status + # ciao-cli -identity=https://ciao-controller.example.com:35357 -username ciao -password ciaoUserPassword -controller=ciao-controller.example.com node status Total Nodes 3 Ready 0 Full 3 @@ -136,8 +136,8 @@ deployment node, which contains the following environment variables: # cat ciaorc export CIAO_CONTROLLER=ciao-controller.example.com export CIAO_IDENTITY=https://ciao-controller.example.com:35357 - export CIAO_USERNAME=csr - export CIAO_PASSWORD=secret + export CIAO_USERNAME=ciao + export CIAO_PASSWORD=ciaoUserPassword then you could verify with the following command: From 2d24ee5e767aadc0f017ca5cd0747f20e6020d87 Mon Sep 17 00:00:00 2001 From: Alberto Murillo Date: Tue, 20 Sep 2016 12:35:14 -0500 Subject: [PATCH 2/6] ciao-deploy.rst: Explain created files The playbook creates some files when its executed. The files are: ./certificates directory which contains the certificates used in the cluster ./images directory which contains the os images used in the cluster ./openrc file which contains environment variables to authenticate openstack cli ./ciaorc file which contains environment variables to authenticate ciao cli Signed-off-by: Alberto Murillo --- source/ciao-deploy.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/source/ciao-deploy.rst b/source/ciao-deploy.rst index 06d04292..2f1ef53c 100644 --- a/source/ciao-deploy.rst +++ b/source/ciao-deploy.rst @@ -111,6 +111,15 @@ be started with the following command: $ ansible-playbook -i hosts ciao.yml --private-key= +Note: The playbook will create the following files in the current folder of the machine runninng the playbooks. + + * ./certificates: This directory contains the certificates that where created and copied to the cluster nodes. + + * ./images: This directory contains the images used by the ciao cluster. (fedora, clearlinux, cnci, ovmf.fd) + + * ./ciaorc: This file contains environment variables needed by ciao cli to authenticate to the ciao cluster. + + * ./openrc: This file contains environment variables needed by openstack cli to authenticate with the ciao cluster. Verify ====== From 366eded708470e9c421e179544d658304e891051 Mon Sep 17 00:00:00 2001 From: Mark Ryan Date: Wed, 21 Sep 2016 10:58:41 +0100 Subject: [PATCH 3/6] ciao-cluster-setup.rst: Update Go requirements. Go 1.7 is now required to build ciao. Signed-off-by: Mark Ryan --- source/ciao-cluster-setup.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/ciao-cluster-setup.rst b/source/ciao-cluster-setup.rst index 7813d94e..c13c3d6f 100644 --- a/source/ciao-cluster-setup.rst +++ b/source/ciao-cluster-setup.rst @@ -84,7 +84,7 @@ bundle, which adds components needed by CIAO:: Build the CIAO software ----------------------- -Ciao is written in the Go programming language. It requires Go 1.6 to +Ciao is written in the Go programming language. It requires Go 1.7 to build. Most Linux distributions have out-of-date versions of Go in their repositories, so you will probably need to download and install a recent version of Go. This can be easily done by following the From cacb450589f1436fee3d0cadd11b07de040483ae Mon Sep 17 00:00:00 2001 From: "L.S. Cook" Date: Wed, 21 Sep 2016 09:29:52 -0700 Subject: [PATCH 4/6] adding CIAO deploy specifics for Ubuntu --- source/ciao-deploy-ubuntu.rst | 218 ++++++++++++++++++++++++++++++++++ 1 file changed, 218 insertions(+) create mode 100644 source/ciao-deploy-ubuntu.rst diff --git a/source/ciao-deploy-ubuntu.rst b/source/ciao-deploy-ubuntu.rst new file mode 100644 index 00000000..5177b805 --- /dev/null +++ b/source/ciao-deploy-ubuntu.rst @@ -0,0 +1,218 @@ +.. _ciao-deploy: + +.. contents:: + +Deploying ciao via automation -- Ubuntu 16.04 +############################################# + +Cloud Integrated Advanced Orchestrator (``ciao``) is a new workload +scheduler designed to address limitations of current cloud OS projects. +Ciao provides a lightweight, fully TLS-based minimal config, is +workload-agnostic, easy to update, offers an optimized-for-speed +scheduler, and is currently optimized for OpenStack*. + +For more information, see https://clearlinux.org/ciao. + +Environment +=========== + +For this example, we'll use a total of five nodes: + - A deployment machine which will be used to run the playbooks. + - A `controller`_ node which will be used to communicate with Keystone. + - Two `compute nodes`_, which will spawn the VMs and containers. + - A `network node`_ which will handle the networking for the workloads. + + Note: The deployment machine is not a necessary part of the cluster; it could be + the sysadmin's computer or a CI/CD server. + +.. _prerequisites: + +Prerequisites +============= + +Ansible* uses :command:`ssh` to run commands on the remote nodes. In +order to do that, the nodes must be configured to allow passwordless SSH +connections from the deployment machine to the cluster nodes. The user +should also have sudo privileges on the cluster nodes. + +The deployment machine needs:: + + Python 2.7 + Ansible >= 2.1 + Golang >= 1.7 + qemu-utils + python-docker + python-openstackclient + Ansible ciao roles from ansible-galaxy + +The managed nodes need:: + + python 2.6 >= + + +Proxy +===== + +If you are running behind a proxy, the following additional steps are needed: + +In the deployment node +---------------------- + +Make sure apt can run. Edit ``/etc/apt/apt.conf`` (to install ansible and +dependencies), appending it with:: + + Acquire::http::Proxy "http://yourproxyaddress:proxyport"; + Install the CIAO roles from ansible-galaxy: + $ sudo -E ansible-galaxy -r requirements --ignore-certs + +In the managed nodes +-------------------- + +Make sure apt can run. Edit ``/etc/apt/apt.conf``, appending it with:: + + Acquire::http::Proxy "http://yourproxyaddress:proxyport"; + +Install Docker and make sure you can pull images. Edit ``/etc/systemd/system/docker.service.d/http-proxy.conf``, appending it with:: + + [Service] + Environment="HTTP_PROXY=http://hostname:port/" + Environment="HTTPS_PROXY=http://hostname:port/" + Environment=”NO_PROXY=localhost,127.0.0.1,.example.com” + +Note: replace the hostname and port of your proxy server and append your local domain name. + +Reload and restart the docker daemon:: + + $ sudo systemctl daemon-reload && sudo systemctl restart docker + + (OPTIONAL): You can download the docker images used in the CIAO deployment. + $ sudo docker pull clearlinux/keystone + $ sudo docker pull clearlinux/ciao-webui + + +Install the software +==================== + +Install ansible from ubuntu packages if the package version is >= 2.1.0; +otherwise, install it from pip (``pip install ansible``). + +Install Dependencies in the deployment machine +---------------------------------------------- + +:: + + python-docker (apt-get) or docker-py (pip) + python-openstackclient (apt-get) + qemu-utils (apt-get) + +Download the ``clear-config-management`` project from github:: + + $ git clone https://github.com/clearlinux/clear-config-management.git + + +Create the playbook +=================== + +The ``clear-config-management`` project includes some sample playbooks that +you may use and customize for your own needs. Start by making a copy of the +sample playbook into your home folder:: + + # cp -r clear-config-management/examples/ciao ~/ + + +Note: These files are also hosted in github + +The relevant files in the playbook are the following: + + * The `ciao.yml`_ file is the master playbook file and includes a playbook + for each component of the cluster. + + * The `hosts`_ file is the hosts inventory file and contains the IP + addresses/FQDN of your nodes, grouped under the roles they will serve + + * The `groups_vars/all`_ file contains variables that will be applied + to your ciao setup. The mandatory variables are already there; be + sure to change the values accordingly to fit your environment + + * The ``ciao_guest_key`` value in :file:`groups_var/all` is the key + to be used to connect to the VMs created by ciao; you can use the + ``ssh-keygen`` command to create one. + +A full list of available variables can be found in the :file:`defaults/main.yml` +file of each role at https://github.com/clearlinux/clear-config-management/tree/master/roles + + +Install the required ansible-roles +================================== + +.. code-block:: console + + # ansible-galaxy install -r requirements.yml + + +Run the playbook +================ +Once you have your variables and hosts file configured, the deployment can +be started with the following command: + +.. code-block:: console + + $ ansible-playbook -i hosts ciao.yml --private-key= + +Note: If you want the latest CIAO changes, change the ``ciao_dev`` variable to ``True`` in the :file:`group_vars/all` file In the ``clear-config-management`` project. + + + + + + +Verify +====== +After ansible is done with the setup, you can verify the cluster is ready +by running the following command on the controller node. Change the **username**, +**password**, **controller**, and **identity** values to match your setup, as +was specified in the ``groups_var/all`` file: + +.. code-block:: console + + # ciao-cli -identity=https://ciao-controller.example.com:35357 \ + -username admin \ + -password secret \ + -controller=ciao-controller.example.com + # node status + Total Nodes 3 + Ready 0 + Full 3 + Offline 0 + Maintenance 0 + +You could also take a look at the :file:`./ciaorc` file created on your +deployment node, which contains the following environment variables: + +.. code-block:: console + + # cat ciaorc + export CIAO_CONTROLLER=ciao-controller.example.com + export CIAO_IDENTITY=https://ciao-controller.example.com:35357 + export CIAO_USERNAME=csr + export CIAO_PASSWORD=secret + +then you could verify with the following command: + +.. code-block:: console + + # source ciaorc + # ciao-cli node status + Total Nodes 3 + Ready 0 + Full 3 + Offline 0 + Maintenance 0 + +.. _controller: https://github.com/01org/ciao/tree/master/ciao-controller +.. _compute nodes: https://github.com/01org/ciao/tree/master/ciao-launcher +.. _network node: https://github.com/01org/ciao/tree/master/ciao-launcher +.. _ciao.yml: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/ciao.yml +.. _hosts: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/hosts +.. _groups_vars/all: https://github.com/clearlinux/clear-config-management/blob/master/examples/ciao/group_vars/all +.. _github: https://github.com/clearlinux/clear-config-management/tree/master/examples/ciao From 93cb76681b18c99e27ee93c325794626ac0d05b9 Mon Sep 17 00:00:00 2001 From: "L.S. Cook" Date: Wed, 21 Sep 2016 10:16:42 -0700 Subject: [PATCH 5/6] fix whitespace and line length Signed-off-by: L.S. Cook --- source/ciao-deploy-ubuntu.rst | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/source/ciao-deploy-ubuntu.rst b/source/ciao-deploy-ubuntu.rst index 5177b805..0c28811f 100644 --- a/source/ciao-deploy-ubuntu.rst +++ b/source/ciao-deploy-ubuntu.rst @@ -72,7 +72,9 @@ Make sure apt can run. Edit ``/etc/apt/apt.conf``, appending it with:: Acquire::http::Proxy "http://yourproxyaddress:proxyport"; -Install Docker and make sure you can pull images. Edit ``/etc/systemd/system/docker.service.d/http-proxy.conf``, appending it with:: +Install Docker and make sure you can pull images. Edit +``/etc/systemd/system/docker.service.d/http-proxy.conf``, appending it +with:: [Service] Environment="HTTP_PROXY=http://hostname:port/" @@ -152,6 +154,7 @@ Install the required ansible-roles Run the playbook ================ + Once you have your variables and hosts file configured, the deployment can be started with the following command: @@ -159,15 +162,14 @@ be started with the following command: $ ansible-playbook -i hosts ciao.yml --private-key= -Note: If you want the latest CIAO changes, change the ``ciao_dev`` variable to ``True`` in the :file:`group_vars/all` file In the ``clear-config-management`` project. - - - - +Note: If you want the latest CIAO changes, change the ``ciao_dev`` +variable to ``True`` in the :file:`group_vars/all` file In the +``clear-config-management`` project. Verify ====== + After ansible is done with the setup, you can verify the cluster is ready by running the following command on the controller node. Change the **username**, **password**, **controller**, and **identity** values to match your setup, as From 0bc8c9451ccaade653564434eaa5791b9e734134 Mon Sep 17 00:00:00 2001 From: John Andersen Date: Thu, 22 Sep 2016 09:17:00 -0700 Subject: [PATCH 6/6] Network boot use latest version Network boot documentation said to set version variable to /usr/share/clear/version and then download https://download.clearlinux.org/current/clear-${version}-pxe.tar.xz however if /usr/share/clear/version does not match https://download.clearlinux.org/latest then clear-${version}-pxe.tar.xz ends up being an xml response saying that the file was not found and the tar step fails. This patch tells the user to set version to https://download.clearlinux.org/latest so that the download from /current/ succeeds. Signed-off-by: John Andersen --- source/network_boot.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/network_boot.rst b/source/network_boot.rst index d9277716..2c73b7f0 100644 --- a/source/network_boot.rst +++ b/source/network_boot.rst @@ -91,7 +91,7 @@ server root ``/var/www/pxe/``. .. code-block:: console # mkdir -p /var/www/pxe/ - # version=$(cat /usr/share/clear/version) + # version=$(curl https://download.clearlinux.org/latest) # curl -o /var/www/pxe/clear-${version}-pxe.tar.xz https://download.clearlinux.org/current/clear-${version}-pxe.tar.xz # tar -xJf /var/www/pxe/clear-${version}-pxe.tar.xz -C /var/www/pxe/ && rm /var/www/pxe/clear-${version}-pxe.tar.xz # unset version