From a3e17f2402b6013afb8773a5da309297ebdca0e7 Mon Sep 17 00:00:00 2001 From: "Tullis, Michael L" Date: Thu, 28 Apr 2016 12:17:47 -0600 Subject: [PATCH] Ceph docs from merge request #55 now going to master branch after a few corrections. --- source/ceph-deploy.rst | 158 +++++++++++++++++++++++++++++++++++++++ source/index-storage.rst | 7 ++ source/index.rst | 14 +++- 3 files changed, 176 insertions(+), 3 deletions(-) create mode 100644 source/ceph-deploy.rst create mode 100644 source/index-storage.rst diff --git a/source/ceph-deploy.rst b/source/ceph-deploy.rst new file mode 100644 index 00000000..2a5bef3b --- /dev/null +++ b/source/ceph-deploy.rst @@ -0,0 +1,158 @@ +Ceph* +##### + +Ceph is a distributed storage system designed to scale well. It can be integrated with +OpenStack* as the storage backend for the OS images and volumes. + +Deploying a Ceph storage cluster is simple using Clear Linux* OS for Intel® Architecture +and Ansible*. + +Environment +=========== +For this example, we'll use a total of five nodes: a **deployment node** which we will use +to run the playbooks, a **monitor node**, and three **storage nodes**. + +Install each component on its own server for best results; however, for testing +purposes you can install the monitor and storage nodes on the same hosts. + +Prerequisites +============= +Ansible uses ssh to run commands on the remote servers. In order to do that, the servers +must be configured to allow passwordless ssh connections from the root user. Follow these +steps to configure your nodes. + +#. Generate ssh keys:: + + # ssh-keygen + +#. Enable root login:: + + # echo "PermitRootLogin yes" >> /etc/ssh/sshd_config + +#. Enable sshd service:: + + # systemctl enable sshd + # systemctl start sshd + +#. Allow passwordless login:: + + # ssh-copy-id root@node + +Install the software +==================== + +Install the ``sysadmin-hostmgmt`` bundle on the development node. This bundle contains +the Ansible software required to run the playbooks, as well as some Ansible roles and +sample playbooks that you can use to build your own:: + + # swupd bundle-add sysadmin-hostmgmt + +Create the playbook +=================== + +The ``sysadmin-hostmgmt`` bundle includes some sample playbooks that you may use and +customize for your own needs. Start by making a copy of the sample playbook into your +home folder.:: + + # cp /usr/share/ansible/examples/ceph ~/ + +The playbook consist of four files that you should modify to fit your needs:: + + ceph + |-- group_vars/ + | |-- all + | +-- mons + |-- hosts + +-- ceph_deploy.yml + +The hosts file contains the IP addresses of your servers grouped under the roles +they will serve:: + + [mons] + 172.28.128.7 + + [osds] + 172.28.128.8 + 172.28.128.9 + 172.28.128.10 + +This :file:`groups_var/all` file contains variables that will applied to all your nodes. +The mandatory variables are already there; be sure to change the values accordingly to +fit your environment. It should look something like this:: + + --- + journal_size: 1024 + monitor_interface: enp0s8 + public_network: 172.28.128.0/24 + cluster_network: "{{ public_network }}" + +A full list of available variables can be found under +:file:`/usr/share/ansible/roles//defaults/main.yml` + +This :file:`groups_var/osd` file contains variables that apply only to the hosts under the +``[mons]`` section in your hosts file. You can choose one of the three available scenarios +for this playbook. + +#. **Journal and osd_data on the same device**: This will co-locate both journal and data + on the same disk, creating a partition at the beginning of the device:: + + journal_collocation: true + devices: + - /dev/sdb + - /dev/sdc + - /dev/sdd + +#. **N journal devices for N OSDs**: In this example, the ``sdb`` partition will be used + for journaling of ``sdc`` and ``sdd sdf`` will be used for journaling of ``sde``:: + + raw_multi_journal: true + devices: + - /dev/sdc + - /dev/sdd + - /dev/sde + raw_journal_devices: + - /dev/sdb + - /dev/sdb + - /dev/sdf + +#. **Specify directory instead of disk for OSDs**:: + + osd_directory: true + osd_directories: + - /var/lib/ceph/osd/mydir1 + - /var/lib/ceph/osd/mydir2 + - /var/lib/ceph/osd/mydir3 + +Run the playbook +================ +Once you have your variables and hosts file configured, the deployment can be fired +with the following command:: + + # ansible-playbook -i hosts ceph_deploy.yml + +Verify +====== +Now that Ansible has finished with the deployment, you can verify the health of the cluster +with the Ceph utilites like ``ceph status`` and ``ceph osd tree``:: + + # ceph status + cluster ee1fae3b-b95b-494c-abd7-f0629d113446 + health HEALTH_OK + monmap e1: 1 mons at {node2=172.28.128.5:6789/0} + election epoch 2, quorum 0 node2 + osdmap e8: 3 osds: 3 up, 3 in + flags sortbitwise + pgmap v14: 64 pgs, 1 pools, 0 bytes data, 0 objects + 7566 MB used, 49647 MB / 59896 MB avail + 64 active+clean +:: + + # ceph osd tree + ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY + -1 0.05699 root default + -2 0.01900 host node3 + 0 0.01900 osd.0 up 1.00000 1.00000 + -3 0.01900 host node4 + 1 0.01900 osd.1 up 1.00000 1.00000 + -4 0.01900 host node5 + 2 0.01900 osd.2 up 1.00000 1.00000 diff --git a/source/index-storage.rst b/source/index-storage.rst new file mode 100644 index 00000000..525bff8b --- /dev/null +++ b/source/index-storage.rst @@ -0,0 +1,7 @@ +Storage solutions +################# + +.. toctree:: + :maxdepth: 1 + + ceph-deploy diff --git a/source/index.rst b/source/index.rst index 3e67c7b2..c8b11093 100644 --- a/source/index.rst +++ b/source/index.rst @@ -1,5 +1,5 @@ .. ClearLinux Documentation documentation master file, created by - sphinx-quickstart on Fri Nov 13 12:23:35 2015. + sphinx-quickstart on Fri Nov 13 12:23:35 2015. You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. @@ -61,6 +61,15 @@ OpenStack* implementation index-openstack.rst +Storage solutions +================= + +.. toctree:: + :maxdepth: 2 + + index-storage.rst + + Indices and tables ================== @@ -70,7 +79,6 @@ Indices and tables License and disclaimers ======================= -.. toctree:: +.. toctree:: .. include:: documentation_license.rst -