Merge branch 'master' into smb

source/clear-linux/tutorials/aws-web/aws-web.rst

@@ -198,13 +198,13 @@ other instances available, they are also listed but not selected.
       chmod 400 AWSClearTestKey.pem
 
 #. Copy the text highlighted in the :guilabel:`Example:` section that is
-   shown in :ref:`figure 11<fig-aws-web-11>`. Paste the copied text into your terminal and add
-   `-l clear` text to the end of the command string. Press the :kbd:`Enter`
-   key to execute the command.
+   shown in :ref:`figure 11<fig-aws-web-11>`. Paste the copied text into your
+   terminal, change the text before the `@` sign to the username `clear`, and
+   press the :kbd:`Enter` key to execute the command.
 
    .. code-block:: console
 
-      ssh -i "AWSClearTestKey.pem" root@ec2-34-209-39-184.us-west-2.compute.amazonaws.com -l clear
+      ssh -i "AWSClearTestKey.pem" clear@ec2-34-209-39-184.us-west-2.compute.amazonaws.com
 
 #. A message appears on the terminal stating the authenticty of the host can't
    be established and prompts you with the message:

source/clear-linux/tutorials/spark.rst

@@ -0,0 +1,134 @@
+  .. _spark:
+
+Set up a standalone cluster system using Apache\* Spark\*
+#########################################################
+
+This tutorial describes how to install, configure, and run Apache Spark on
+|CLOSIA|. Apache Spark is a fast general-purpose cluster computing system with
+the following features:
+
+*  Provides high-level APIs in Java\*, Scala\*, Python\*, and R\*.
+*  Includes an optimized engine that supports general execution graphs.
+*  Supports high-level tools including Spark SQL, MLlib, GraphX, and Spark
+   Streaming.
+
+In this tutorial, you will install Spark on a single machine running the
+master daemon and a worker daemon.
+
+Prerequisites
+*************
+
+This tutorial assumes you have installed |CL| on your host system.
+For detailed instructions on installing |CL| on a bare metal system, visit
+the :ref:`bare metal installation tutorial<bare-metal-install>`.
+
+Before you install any new packages, update |CL| with the following command:
+
+.. code-block:: bash
+
+   sudo swupd update
+
+Install Apache Spark
+********************
+
+Apache Spark is included in the :file:`big-data-basic` bundle. To install the
+framework, enter:
+
+.. code-block:: bash
+
+   sudo swupd bundle-add big-data-basic
+
+Configure Apache Spark
+**********************
+
+#. Create the configuration directory with the command:
+
+   .. code-block:: bash
+
+      sudo mkdir /etc/spark
+
+#. Copy the default templates from :file:`/usr/share/defaults/spark` to
+   :file:`/etc/spark` with the command:
+
+   .. code-block:: bash
+
+      sudo cp /usr/share/defaults/spark/* /etc/spark
+
+   .. note:: Since |CL| is a stateless system, you should never modify the
+      files under the :file:`/usr/share/defaults` directory. The software
+      updater overwrites those files.
+
+
+#. Copy the template files below to create custom configuration files:
+
+   .. code-block:: bash
+
+      sudo cp /etc/spark/spark-defaults.conf.template /etc/spark/spark-defaults.conf
+      sudo cp /etc/spark/spark-env.sh.template /etc/spark/spark-env.sh
+      sudo cp /etc/spark/log4j.properties.template /etc/spark/log4j.properties
+
+#. Edit the :file:`/etc/spark/spark-env.sh` file and add the
+   :envvar:`SPARK_MASTER_HOST` variable. Replace the example address below
+   with your localhost IP address. View your IP address using the
+   :command:`hostname -I` command.
+
+   .. code-block::
+
+      SPARK_MASTER_HOST="10.300.200.100"
+
+   .. note:: This optional step enables the master's web user interface to
+      view information needed later in this tutorial.
+
+#. Edit the :file:`/etc/spark/spark-defaults.conf` file and update the
+   `spark.master` variable with the `SPARK_MASTER_HOST` address and port `7077`.
+
+   .. code-block::
+
+      spark.master    spark://10.300.200.100:7077
+
+Start the master server and a worker daemon
+*******************************************
+
+#. Start the master server using:
+
+   .. code-block:: bash
+
+      sudo /usr/share/apache-spark/sbin/./start-master.sh
+
+#. Start one worker daemon and connect it to the master using the
+   `spark.master` variable defined earlier:
+
+   .. code-block:: bash
+
+      sudo /usr/share/apache-spark/sbin/./start-slave.sh spark://10.300.200.100:7077
+
+#. Open an internet browser and view the worker daemon information using
+   the master's IP address and port `8080`:
+
+   .. code-block::
+
+      http://10.300.200.100:8080
+
+Run the Spark wordcount example
+*******************************
+
+#. Run the wordcount example using a file on your local host and output the
+   results to a new file with the following command:
+
+   .. code-block:: bash
+
+      sudo spark-submit /usr/share/apache-spark/examples/src/main/python/wordcount.py ~/Documents/example_file > ~/Documents/results
+
+#. Open an internet browser and view the application information using
+   the master's IP address and port `8080`:
+
+   .. code-block::
+
+      http://10.300.200.100:8080
+
+#. View the results of the wordcount application in the :file:`~/Documents/results` file.
+
+**Congratulations!**
+
+You successfully installed and set up a standalone Apache Spark cluster.
+Additionally, you ran a simple wordcount example.

source/clear-linux/tutorials/telemetry-backend/telemetry-backend.rst

@@ -13,6 +13,7 @@ also redirect where records go if they wish to collect records for themselves
 by setting up and using their own telemetry backend server.
 
 A telemetry backend server consists of two Flask applications:
+
 * The :guilabel:`collector` is an ingestion app for records received from the
   :guilabel:`telemetrics-client` probes.
 * The :guilabel:`telemetryui` web app exposes several visualizations of the

source/clear-linux/tutorials/tutorials.rst

@@ -18,4 +18,5 @@ specific |CLOSIA| use cases.
    fmv
    aws-web/aws-web
    telemetry-backend/telemetry-backend
-   smb-file-sharing/smb-file-sharing
+   smb/smb
+   spark