mirror of
https://github.com/clearlinux/clear-linux-documentation.git
synced 2026-07-04 19:56:02 +00:00
added instructions for creating backend server
This commit is contained in:
Beth Dean
@@ -2,40 +2,321 @@
|
||||
|
||||
|
||||
|CL-ATTR| includes a telemetry and analytics solution (also known as telemetrics) as part of the OS, which records events of interest and reports them back to the development team using the telemetrics client daemons.
|
||||
End users can enable or disable the telemetry client component of |CL| and also redirect where records are sent if they wish to collect records for themselves by using their own telemetry backend server.
|
||||
End users can enable or disable the telemetry client component of |CL| and also redirect where records are sent if they wish to collect records for themselves by using their own telemetry backend server. More detailed information about using and configuring the telemetrics client is found in the :ref:`telemetrics` guide.
|
||||
|
||||
This tutorial walks you through setting up a telemetry backend server to manage your records, and using the telemetry API to add telemetry to your own applications.
|
||||
This tutorial walks you through setting up a telemetry backend server to manage your records, and how to use the telemetry API to add telemetry to your own applications.
|
||||
|
||||
|
||||
Prerequisites
|
||||
=============
|
||||
|
||||
For this tutorial, you can use an existing |CL| system, or you can start with a clean installation of |CL| on a new system
|
||||
using the :ref:`bare-metal-install` getting started guide and:
|
||||
For this tutorial, you can use an existing |CL| system, or you can start with a clean installation of |CL| on a new system.
|
||||
|
||||
#. Choose the automatic install.
|
||||
New Installation
|
||||
****************
|
||||
|
||||
To setup a new system for your telemetry backend server, follow the :ref:`bare-metal-install` getting started guide and:
|
||||
|
||||
#. Choose to install |CL|.
|
||||
#. Join the :guilabel:`Stability Enhancement Program` during the installation process to enable the telemetrics client components.
|
||||
#. Select the manual installation method with the following settings:
|
||||
* Set the hostname to :guilabel:`clr-telem-server`
|
||||
* Create an administrative user named :guilabel:`clear` and add this user to sudoers :ref:`enable-user-space`
|
||||
* Choose the :file:`dev-utils`, :file:`network-basic`, and :file:`openssh-server` bundles from the bundle list
|
||||
|
||||
If you are using an existing |CL| system, make sure you have installed the telemetry bundle. Use the :command:`swupd` utility with the `bundle-list` option and check for "telemetrics" in the list:
|
||||
|
||||
.. note::
|
||||
|
||||
Bundles can also be added to your system after this install process completed. The bundles listed here are a minimal set needed to complete the setup of the telemetry backend server and applications.
|
||||
|
||||
Existing System
|
||||
***************
|
||||
|
||||
If you are using an existing |CL| system, make sure you have installed the telemetry and dev-utils bundles. Use the :command:`swupd` utility with the `bundle-list` option and check for "telemetrics" in the list:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd bundle-list
|
||||
sudo swupd bundle-list
|
||||
|
||||
If you need to install the telemetrics bundle, use :command:`swupd` to do so.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo swupd bundle-add telemetrics
|
||||
sudo swupd bundle-add telemetrics
|
||||
|
||||
More information about enabling and configuring the telemetry client can be found at :ref:`telemetry-enable`.
|
||||
|
||||
Setting up the telemetry backend server
|
||||
=======================================
|
||||
We'll be using the :file:`deploy.sh` file from the `clearlinux/telemetrics-backend`_ Git repository to install required dependencies for the web server applications. The script also configures nginx and uwsgi, deploys snapshots of the applications, and starts all required services.
|
||||
|
||||
To set up a telemetry backend server you can use the same |CL| system we're working with for this tutorial, or setup on a separate system. Follow the :ref:`telemetry-backend` guide to setup the server and redirect your telemetry records to it. Refer to the :ref:`telemetry-config` guide to configure the telemetry client and specify where to send the records.
|
||||
Download the clearlinux/telemetrics-backend Git repository
|
||||
**********************************************************
|
||||
|
||||
With all prerequisite software bundles installed, log in with your administrative user, and from your :file:`$HOME` directory, run :command:`git` to clone the :guilabel:`telemetrics-backend` repository into the :file:`$HOME/telemetrics-backend` directory:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone https://github.com/clearlinux/telemetrics-backend
|
||||
|
||||
.. note::
|
||||
|
||||
You may need to set up the :envvar:`https_proxy` environment variable if you have issues reaching github.com.
|
||||
|
||||
Run the deploy.sh script to install the backend server
|
||||
******************************************************
|
||||
|
||||
Change your current working directory to :file:`telemetrics-backend/scripts`.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
cd telemetrics-backend/scripts
|
||||
|
||||
Run the :command:`./deploy.sh -h` to see the list of options for the :command:`deploy.sh` script:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
./deploy.sh -h
|
||||
Deploy snapshot of the telemetrics-backend
|
||||
|
||||
-a Perform specified action (deploy, install, migrate, resetdb,
|
||||
restart, uninstall; default: deploy)
|
||||
-d Distro to deploy to (ubuntu, centos or clr; default: ubuntu)
|
||||
-h Print these options
|
||||
-H Set domain for deployment (only accepted value is "localhost" for
|
||||
now)
|
||||
-r Set repo location to deploy from
|
||||
(default: https://github.com/clearlinux/telemetrics-backend)
|
||||
-s Set source location (default: "master" branch from git repo)
|
||||
-t Set source type (tarball, or git; default: git)
|
||||
-u Perform complete uninstallation
|
||||
|
||||
The :command:`deploy.sh` is a bash shell script that allows you to perform the following actions:
|
||||
|
||||
* *deploy* - install a complete instance of the telemetrics backend server and all required components. This is the default action if no *-a* argument is given on the command line.
|
||||
* *install* - installs and enables all required components for the telemetrics backend server.
|
||||
* *migrate* - migrate database to new schema.
|
||||
* *resetdb* - reset the database.
|
||||
* *restart* - restart the nginx and uWSGI services.
|
||||
* *uninstall* - uninstall all packages.
|
||||
|
||||
.. note::
|
||||
|
||||
The *uninstall* option does not perform any actions if the distro is set to |CL| and will only uninstall packages if the distro is Ubuntu
|
||||
|
||||
Next, we will install the telemetrics backend server with the following options:
|
||||
|
||||
* *-a install* to perform an install
|
||||
* *-d clr* to install to a |CL| distro
|
||||
* *-H localhost* to set the domain to localhost
|
||||
|
||||
We do not need to set the following options since the values are set to the correct values we want by default:
|
||||
|
||||
* *-r https://github.com/clearlinux/telemetrics-backend* sets the repo location for :command:`git` to clone from.
|
||||
* *-s master* to set the location, or branch.
|
||||
* *-t git* to set the source type to git.
|
||||
|
||||
.. caution::
|
||||
The :file:`deploy.sh` shell script has minimal error checking and makes several changes to your system. Be sure that the options you define on the cmdline are correct before proceeding.
|
||||
|
||||
To begin the installation with the options defined:
|
||||
|
||||
Run the shell script from the :file:`$HOME/telemetrics-backend/scripts` directory:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./deploy.sh -H localhost -a install -d clr
|
||||
|
||||
The script will start and list all the defined options and prompt you for the :guilabel:`PostgreSQL` database password as shown below:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
Options:
|
||||
host: localhost
|
||||
distro: clr
|
||||
action: install
|
||||
repo: https://github.com/clearlinux/telemetrics-backend
|
||||
source: master
|
||||
type: git
|
||||
DB password: (default: postgres):
|
||||
|
||||
For the :guilabel:`DB password:`, press the :kbd:`Enter` key to accept the default password `postgres`.
|
||||
|
||||
The :command:`swupd` begins installing the required software bundles to set up the telemetrics backend server. The output will look similar to the following:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
swupd-client bundle adder 3.12.7
|
||||
Copyright (C) 2012-2017 Intel Corporation
|
||||
|
||||
Downloading packs...
|
||||
|
||||
Extracting application-server pack for version 18740
|
||||
...5%
|
||||
Extracting database-basic-dev pack for version 18670
|
||||
...10%
|
||||
Extracting database-basic pack for version 18670
|
||||
...15%
|
||||
Extracting os-clr-on-clr pack for version 18740
|
||||
...21%
|
||||
Extracting sysadmin-basic-dev pack for version 18740
|
||||
...26%
|
||||
Extracting storage-utils-dev pack for version 18770
|
||||
...31%
|
||||
Extracting os-core-update-dev pack for version 18760
|
||||
...36%
|
||||
Extracting network-basic-dev pack for version 18760
|
||||
...42%
|
||||
Extracting mixer pack for version 18790
|
||||
...47%
|
||||
Extracting os-installer pack for version 18800
|
||||
...52%
|
||||
Extracting mail-utils-dev pack for version 18760
|
||||
...57%
|
||||
Extracting koji pack for version 18800
|
||||
...63%
|
||||
Extracting go-basic pack for version 18800
|
||||
...68%
|
||||
Extracting dev-utils-dev pack for version 18820
|
||||
...73%
|
||||
Extracting python-basic-dev pack for version 18750
|
||||
...78%
|
||||
Extracting perl-basic-dev pack for version 18610
|
||||
...84%
|
||||
Extracting c-basic pack for version 18800
|
||||
...89%
|
||||
Extracting os-core-dev pack for version 18800
|
||||
...94%
|
||||
Extracting web-server-basic pack for version 18680
|
||||
...100%
|
||||
Installing bundle(s) files...
|
||||
...100%
|
||||
Calling post-update helper scripts.
|
||||
Possible filedescriptor leak : 8 (socket:[30833])
|
||||
Bundle(s) installation done.
|
||||
|
||||
.. note::
|
||||
|
||||
This script uses :command:`sudo` to run commands and you may be prompted to enter your user password at any time while the script is executing. If this occurs, enter your user password to execute the :command:`sudo` command.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
Password:
|
||||
|
||||
|
||||
You may also see an informational message about setting the :envvar:`https_proxy` environment variable if this variable isn't set.
|
||||
|
||||
|
||||
Once the :command:`swupd` command is complete, the script begins processing the requirements to install and implement the telemetrics server. Finally, the script enables the server and provides output that finishes with something similar to:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
.
|
||||
.
|
||||
Successfully built alembic Flask-Migrate itsdangerous Mako MarkupSafe python-editor SQLAlchemy uWSGI WTForms
|
||||
Installing collected packages: SQLAlchemy, MarkupSafe, Mako, python-editor, six, python-dateutil, alembic, click, Werkzeug, Jinja2, itsdangerous, Flask, Flask-SQLAlchemy, Flask-Migrate, WTForms, Flask-WTF, psycopg2, uWSGI
|
||||
Running setup.py install for psycopg2 ... done
|
||||
Successfully installed Flask-0.12.2 Flask-Migrate-2.1.0 Flask-SQLAlchemy-2.2 Flask-WTF-0.14.2 Jinja2-2.9.6 Mako-1.0.7 MarkupSafe-1.0 SQLAlchemy-1.1.13 WTForms-2.1 Werkzeug-0.12.2 alembic-0.9.5 click-6.7 itsdangerous-0.24 psycopg2-2.7.3 python-dateutil-2.6.1 python-editor-1.0.3 six-1.10.0 uWSGI-2.0.15
|
||||
|
||||
Once all the server components have been installed you are prompted to enter the :guilabel:`PostgreSQL` database password to change it as illustrated below:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
Enter password for 'postgres' user:
|
||||
New password:
|
||||
Retype new password:
|
||||
passwd: password updated successfully
|
||||
|
||||
|
||||
Enter `postgres` for the current value of the password and then enter a new password, retype it to verify the new password and the :guilabel:`PostgreSQL` database password will be updated.
|
||||
|
||||
The script finalizes installation and finishes.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service → /usr/lib/systemd/system/postgresql.service.
|
||||
Cloning into 'telemetrics-backend'...
|
||||
remote: Counting objects: 344, done.
|
||||
remote: Compressing objects: 100% (53/53), done.
|
||||
remote: Total 344 (delta 30), reused 50 (delta 20), pack-reused 268
|
||||
Receiving objects: 100% (344/344), 130.20 KiB | 1.40 MiB/s, done.
|
||||
Resolving deltas: 100% (177/177), done.
|
||||
.
|
||||
.
|
||||
.
|
||||
Already using interpreter /usr/bin/python3
|
||||
Using base prefix '/usr'
|
||||
New python executable in /var/www/telemetry/venv/bin/python3
|
||||
Not overwriting existing python script /var/www/telemetry/venv/bin/python (you must use /var/www/telemetry/venv/bin/python3)
|
||||
Installing setuptools, pip, wheel...done.
|
||||
INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
|
||||
INFO [alembic.runtime.migration] Will assume transactional DDL.
|
||||
INFO [alembic.runtime.migration] Running upgrade -> 3230c615d6e0, empty message
|
||||
INFO [alembic.runtime.migration] Running upgrade 3230c615d6e0 -> 466cf2f35d67, empty message
|
||||
|
||||
Install complete (installation folder: /var/www/telemetry)
|
||||
|
||||
Once the installation is complete you can use your web browser and view the new server by opening the web browser on your system and type in ``localhost`` in the address bar.
|
||||
|
||||
You should see a web page similar to the one shown in figure 1:
|
||||
|
||||
.. TODO fix links for figures
|
||||
.. figure:: telemetry-backend/figures/telemetry-backend-1.png
|
||||
:scale: 50 %
|
||||
:alt: Telemetry UI
|
||||
|
||||
Figure 1: :guilabel:`Telemetry UI`
|
||||
|
||||
Redirect telemetry records
|
||||
**************************
|
||||
|
||||
Telemetry records generated by the telemetrics clients are sent to the server location defined in the :file:`/usr/share/defaults/telemetrics/telemetrics.conf` configuration file. You can customize this setting by copying this file to :file:`/etc/telemetrics/telemetrics.conf` and changing the ``server=`` setting to your new server location.
|
||||
|
||||
#. Create the :file:`/etc/telemetrics` directory and make it your current working directory.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo mkdir -p /etc/telemetrics
|
||||
cd /etc/telemetrics
|
||||
|
||||
|
||||
#. Copy the default :file:`telemetrics.conf` file to the new :file:`/etc/telemetrics` directory.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo cp /usr/share/defaults/telemetrics/telemetrics.conf
|
||||
|
||||
#. Edit the new :file:`/etc/telemetrics/telemetrics.conf` file with your editor using the :command:`sudo` directive and change the :guilabel:`server=` setting to ``http://localhost/v2/collector`` and save this change in the new file.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
server=http://localhost/v2/collector
|
||||
|
||||
You can also use the fully qualified domain name for your server instead of :guilabel:`localhost`.
|
||||
|
||||
#. Restart the telemetry daemons to reload the configuration file.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
telemctl restart
|
||||
|
||||
Test the new telemetry backend server
|
||||
*************************************
|
||||
|
||||
|CL| includes a telemetry test probe called :command:`hprobe` that will send a ``hello`` record to the telemetry backend server. To test that the telemetry records are now going to your new destination, run the :command:`hprobe` command to send a ``hello`` record to the server as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
hprobe
|
||||
|
||||
The record should show up on your new server console as shown in figure 2:
|
||||
|
||||
.. figure:: telemetry-backend/figures/telemetry-backend-2.png
|
||||
:scale: 50 %
|
||||
:alt: Telemetry UI
|
||||
|
||||
Figure 2: :guilabel:`Telemetry UI`
|
||||
|
||||
Once the telemetry backend server is running and/or you have enabled local retention of the records, and you have verified that your client is enabled and sending records to the server by using the :command:`hprobe` utility, you're ready to begin adding custom telemetry events to your applications.
|
||||
|
||||
Creating custom telemetry events
|
||||
================================
|
||||
@@ -300,9 +581,11 @@ To verify that the heartbeat message was received by the telemetry backend serve
|
||||
A full example of the `heartbeat probe`_ in C is documented in the source code. For more information about telemetrics in |CL| refer to the :ref:`telemetrics` guide.
|
||||
|
||||
|
||||
You can also look for the record on the telemetry backend server.
|
||||
You can also look for the record on the telemetry backend server.
|
||||
|
||||
.. _latest version:
|
||||
https://github.com/clearlinux/telemetrics-client/tree/master/src
|
||||
|
||||
.. _heartbeat probe: https://github.com/clearlinux/telemetrics-client/tree/master/src/probes/hello.c
|
||||
|
||||
.. _clearlinux/telemetrics-backend: https://github.com/clearlinux/telemetrics-backend
|
||||
|
||||
Reference in New Issue
Block a user