diff --git a/source/ciao-cluster-setup.rst b/source/ciao-cluster-setup.rst index 0b5d512f..23b1d4bc 100644 --- a/source/ciao-cluster-setup.rst +++ b/source/ciao-cluster-setup.rst @@ -2,8 +2,8 @@ .. contents:: -Getting started with ciao -######################### +ciao cluster setup +################## This topic explains how to set up a cluster of machines running Clear Linux* OS for Intel® Architecture with :abbr:`Cloud Integrated Advanced Orchestrator (CIAO)`, or ciao. @@ -22,7 +22,7 @@ You'll need at least four machines and a switch connecting them to form your beginning ciao cluster. The switch is assumed to be plugged directly into an "upstream" network running a DHCP server. See the illustration below as an example: -.. image:: _static/images/image-blob-ciao-networking.png +.. image:: image-blob-ciao-networking.png The following examples assume you have four nodes on a ``192.168.0.0/16`` network: @@ -37,21 +37,21 @@ Network node ("nn") ~~~~~~~~~~~~~~~~~~~ * IP ``192.168.0.102`` -* Runs Launcher +* Runs Launcher with ``--network=nn`` option * Has CNCI image in ``/var/lib/ciao/images``. See below for more on CNCI image preparation. Compute node 1 ("cn1") ~~~~~~~~~~~~~~~~~~~~~~ * IP ``192.168.0.103`` -* Runs Launcher +* Runs Launcher with ``--network=cn`` option * Has workload images in ``/var/lib/ciao/images`` Compute node 2 ("cn2") ~~~~~~~~~~~~~~~~~~~~~~ * ``IP 192.168.0.104`` -* Runs Launcher +* Runs Launcher with ``--network=cn option`` * Has workload images in ``/var/lib/ciao/images`` @@ -75,13 +75,13 @@ OS on all nodes by following the instructions in the topic `downloadable installer images`_ are compatible with ciao. After the base installation on each node add the following additional -bundle, which adds components needed by ciao:: +bundle, which adds components needed by CIAO:: $ swupd bundle-add cloud-control -Build the ciao software +Build the CIAO software ----------------------- Ciao is written in the Go programming language. It requires Go 1.6 to @@ -95,10 +95,10 @@ present, use the ``go get`` tool to fetch and build ciao and its go dependencies:: $ cd $GOPATH/src - $ go get -v -u github.com/01org/ciao/... + $ go get -v -u github.com/01org/ciao The binaries will install to ``$GOPATH/bin``. You should have -``ciao-cli``, ``ciao-cert``, ``ciao-cnci-agent``, ``ciao-launcher``, +``ciao-cli``, ``ciao-cert``, ``cnci_agent``, ``ciao-launcher``, ``ciao-controller``, and ``ciao-scheduler``. Build certificates @@ -117,7 +117,7 @@ SSNTP server, and connecting clients will validate credentials matched by those embedded in the certificates. Create unique certificates for each of your scheduler, compute node, network -node launchers, cnci agent, controller, and the CNCI launcher; save each with a +node launchers, cnciagent, controller, and the CNCI launcher; save each with a unique name. The names, locations, and contents (signer and role) of the certificates are very important. The rest of this topic will consistently use the following example filenames: @@ -146,10 +146,9 @@ the `Starting a workload` section. Keystone node ------------- -You need to run a Keystone service. General documentation on -`installing Keystone`_ and -`setting up Keystone`_ services can be found at the OpenStack developer website. -Once your Keystone is running, we need a few configuration points. For example::: +You need to run a Keystone service. General documentation on setting +up Keystone services can be found at the `OpenStack developer`_ website. +We need a few configuration points. For example::: $ openstack service create --name ciao compute $ openstack user create --password hello csr @@ -168,35 +167,6 @@ The controller node will host your controller and scheduler. Certificates are as to be in ``/etc/pki/ciao``, generated with the correct roles and names as previously described. -Cluster Configuration -~~~~~~~~~~~~~~~~~~~~~ -Ciao's cluster configuration is stored and fetched from a cluster specific storage backend. -Supported backends are plain **local file**, **etcd** [WIP] and **ZooKeeper** [WIP]. - -For more details about Cluster Configuration Architecture: `CIAO Configuration Architecture`_ - -- Local File backend - - - Create the ``/etc/ciao/configuration.yaml`` file. Example:: - - configure: - scheduler: - storage_uri: /etc/ciao/configuration.yaml - controller: - compute_ca: /etc/pki/ciao/compute_ca.pem - compute_cert: /etc/pki/ciao/compute_key.pem - identity_user: controller - identity_password: ciao - launcher: - compute_net: 192.168.1.110 - mgmt_net: 192.168.1.111 - image_service: - url: http://glance.example.com - identity_service: - url: http://keystone.example.com - - - More examples at: `CIAO Configuration examples`_ - Scheduler ~~~~~~~~~ @@ -269,9 +239,12 @@ images with which you wish to test. Start the compute node launcher ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The launcher is run with options for declaring certificates. For example:: +The launcher is run with options declaring certificates, maximum VMs +(controls when FULL is returned by a node, scaling to the resources +available on your node), server location, and compute node ("cn") +launching type. For example:: - $ sudo ./ciao-launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-CNAgent-localhost.pem + $ sudo ./launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-CNAgent-localhost.pem --server= --network=cn --compute-net --mgmt-net Optionally, add ``-logtostderr`` (more verbose with also ``-v=2``) to get console logging output. @@ -296,17 +269,19 @@ This section describes how to generate a CNCI image from a vanilla Clear Cloud qcow2 image:: $ cd /var/lib/ciao/images - $ curl -O https://download.clearlinux.org/demos/ciao/clear-8260-ciao-networking.img.xz - $ xz -T0 --decompress clear-8260-ciao-networking.img.xz - $ ln -s clear-8260-ciao-networking.img 4e16e743-265a-4bf2-9fd1-57ada0b28904 - $ $GOPATH/src/github.com/01org/ciao/networking/ciao-cnci-agent/scripts/update_cnci_cloud_image.sh /var/lib/ciao/images/clear-8260-ciao-networking.img /etc/pki/ciao/ + $ curl -O https://download.clearlinux.org/demos/ciao/clear-7470-ciao-networking.img.xz + $ xz -T0 --decompress clear-7470-ciao-networking.img.xz + $ ln -s clear-7470-ciao-networking.img 4e16e743-265a-4bf2-9fd1-57ada0b28904 + $ $GOPATH/src/github.com/01org/ciao/networking/cnci_agent/scripts/update_cnci_cloud_image.sh /var/lib/ciao/images/clear-7470-ciao-networking.img /etc/pki/ciao/ Start the network node launcher ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The network node's launcher is run similarly to the compute node's launcher:: +The network node's launcher is run similarly to the compute node's launcher. +The primary difference is that it uses the network node ("nn") launching +type:: - $ sudo ./ciao-launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-NetworkingAgent-localhost.pem + $ sudo ./ciao-launcher --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-NetworkingAgent-localhost.pem --server= --network=nn --compute-net --mgmt-net Start the controller -------------------- @@ -320,20 +295,14 @@ and network node already up and running together.** the correct roles and names as previously described. #. Copy in the initial database table data from the ciao-controller source - (``$GOPATH/src/github.com/01org/ciao/ciao-controller/tables`` on your - build/development) to ``./tables`` in the same directory as the - ciao-controller binary. Copying in ``*.csv`` will work if you are testing - a Clear Cloud image, Fedora image and Docker. Other images will require - edits to the csv config files. This location is the default, but can be - changed by modifying the controller command line to include - ``--tables_init_path=``. + (``$GOPATH/src/github.com/01org/ciao/ciao-controller`` on your + build/development) to the same directory as the ciao-controller binary. + Copying in ``*.csv`` will work if you are testing a Clear Cloud image, + Fedora image and Docker. Other images will require edits to the csv + config files. -#. Copy in the test workload definitions from - ``$GOPATH/src/github.com/01org/ciao/ciao-controller/workloads`` on your - build/development machine to ``./workloads`` in the same directory as the - ciao-controller binary. This location is the default, but can be changed - by modifying the controller command line to include - ``--workloads_path=``. +#. Copy in the test.yaml file from + ``$GOPATH/src/github.com/01org/ciao/ciao-controller/test.yaml``. The `ciao-controller workload_resources.csv`_ and the `ciao-controller workload_template.csv`_ have four stanzas, so yours @@ -343,6 +312,10 @@ CNCI). To run other images of your choosing, follow a process similar to the above: pre-populate OS images and edit each of these two files on your controller node. +If the controller is on the same physical machine as the scheduler, the +``--url`` option is optional; otherwise it refers to your scheduler +SSNTP server IP. + In order for the ciao-controller's go code to correctly use the CA certificate(s) generated earlier when you built your keystone server, this certificate needs to be installed in the control node and be @@ -367,7 +340,7 @@ name of the system that is hosting the keystone service**. An SSL-enabled Keystone is required, with additional parameters for ciao-controller pointing at its certificates:: - $ sudo ./ciao-controller --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-Controller-localhost.pem + $ sudo ./ciao-controller --cacert=/etc/pki/ciao/CAcert-[scheduler-node-hostname].pem --cert=/etc/pki/ciao/cert-Controller-localhost.pem -identity=https://[keystone-FQDN]:35357 --username= --password= --url --httpskey=./key.pem --httpscert=./cert.pem Optionally add ``-logtostderr`` (more verbose with also ``-v=2``) to get console logging output. @@ -375,17 +348,17 @@ console logging output. Use the `ciao-cli`_ command line tool to verify that your cluster is now up and running:: - $ ciao-cli -username admin -password node status - $ ciao-cli -username admin -password node list -compute - $ ciao-cli -username admin -password node list -cnci + $ ciao-cli -username admin -password -cluster-status + $ ciao-cli -username admin -password -list-cns + $ ciao-cli -username admin -password -list-cncis -``node status`` shows the number of nodes in your cluster, and the +``-cluster-status`` shows the number of nodes in your cluster, and the status of each. -``node list -compute`` displays a more detailed view (number of instances per node, +``-list-cns`` displays a more detailed view (number of instances per node, available resources per node, etc.). -``node list -cnci`` provides information about the current CNCI VMs, and their statuses. +``-list-cncis`` provides information about the current CNCI VMs, and their statuses. Interacting with your cluster ============================= @@ -441,36 +414,53 @@ As a valid user, the `ciao-cli`_ tool allows you to start a workload. First, you may want to know which workloads are available:: - $ ciao-cli workload list + $ ciao-cli -list-workloads Then you can launch one or more workloads:: - $ ciao-cli instance add -workload -instances + $ ciao-cli -launch-instances -workload -instances And you can monitor all your instances statuses (``pending`` or ``running``):: - $ ciao-cli instance list + $ ciao-cli -list-instances Performance data can be obtained (optionally) by adding a specific label to all your instances:: - $ ciao-cli instance add -label -workload -instances + $ ciao-cli -launch-instances -instance-label -workload -instances And eventually fetch the performance data:: - $ ciao-cli trace show