diff --git a/source/clear-linux/guides/maintenance/autospec.rst b/source/clear-linux/guides/maintenance/autospec.rst index 6e7aa873..cc22ad33 100644 --- a/source/clear-linux/guides/maintenance/autospec.rst +++ b/source/clear-linux/guides/maintenance/autospec.rst @@ -18,9 +18,12 @@ Description The autospec tool attempts to infer the requirements of the :file:`.spec` file by analyzing the source code and :file:`Makefile` information. It will continuously run updated builds based on new information discovered from build -failures until it has a complete and valid :file:`.spec` file. The autospec tool -makes use of mock to achieve this. It is possible to influence the exact -behavior of autospec by providing `control files`_. +failures until it has a complete and valid :file:`.spec` file. You can influence +the behavior of autospec and customize the build by providing `control files`_ +to the autospec tool. + +autospec uses mock as a sandbox to run the builds. Visit the `mock wiki`_ for +additional information on using mock. For a general understanding of how RPMs work, visit the `rpm website`_ or the `RPM Packaging Guide`_ . @@ -49,7 +52,7 @@ Prerequisites Workflow ======== -First time setup +First-time setup ---------------- Before you use autospec, you will need to set up the autospec environment and @@ -62,15 +65,20 @@ The `user-setup script`_ creates the autospec workspace in the contains the main tools, `autospec` and `common`, used for making packages in |CL|. -Create a RPM ------------- +Create an RPM +------------- The basic autospec process is described in the following steps: #. The :command:`make autospec` command generates a :file:`.spec` file based on - analysis of code and control files, if present. + analysis of code and existing control files. -#. autospec creates a ``build root`` with mock config. + Control files should be located in the same directory as the resulting + :file:`.spec` file. + + View the `autospec README`_ for more information on `control files`_. + +#. autospec creates a build root with mock config. #. autospec attempts to build an RPM from the generated :file:`.spec`. @@ -82,15 +90,17 @@ The basic autospec process is described in the following steps: #. If autospec detects the root cause and knows how to continue, it will restart the build automatically at step 1 with updated build instructions. -#. Otherwise, autospec will stop the build for user inspection and editing of - control files to resolve the errors. The user resumes the process at step 1 - after errors are resolved. +#. Otherwise, autospec will stop the build for user inspection to resolve the + errors. Respond to the build process output by fixing source code issues + and/or editing control files to resolve issues, which may include + dependencies or exclusions. See `autospec README`_ for more information on + control files. + + The user resumes the process at step 1 after errors are resolved. If a binary dependency doesn't exist in |CL|, you will need to build it before running autospec again. - View the `autospec README`_ for more information on control files. - Following these steps, autospec continues to rebuild the package, based on new information discovered from build failures, until it has a valid :file:`.spec`. If no build errors occur, RPM packages are successfully built. @@ -109,19 +119,19 @@ Example 1: First time setup This example shows the basic steps for first time setup of autospec. -#. Download the :file:`user-setup.sh` script. +#. Download the :file:`user-setup.sh` script: .. code-block:: bash curl -O https://raw.githubusercontent.com/clearlinux/common/master/user-setup.sh -#. Make :file:`user-setup.sh` executable. +#. Make :file:`user-setup.sh` executable: .. code-block:: bash chmod +x user-setup.sh -#. Run the script as an unprivileged user. +#. Run the script as an unprivileged user: .. code-block:: bash @@ -130,7 +140,7 @@ This example shows the basic steps for first time setup of autospec. #. After the script completes, log out and log in again to complete the setup process. -#. Set your Git user email and username for the repos on your system +#. Set your Git user email and username for the repos on your system: .. code-block:: bash @@ -146,18 +156,25 @@ an existing spec file. The example uses the ``dmidecode`` package. Make sure you have completed `Example 1: First time setup`_ before starting this example. -#. Navigate to the autospec workspace and clone the ``dmidecode`` package. +#. Navigate to the autospec workspace and clone the ``dmidecode`` package: .. code-block:: bash cd ~/clearlinux make clone_dmidecode - Or you can clone all package repos at once using: + .. note:: - .. code-block:: bash + You can clone all package repos at once using: - make [-j NUM] clone-packages + .. code-block:: bash + + make [-j NUM] clone-packages + + The optional NUM is the number of threads to use. + + For a list of available packages, view the + :file:`~/clearlinux/projects/common/packages` file. #. Navigate to the local copy of the ``dmidecode`` package and build it: @@ -166,7 +183,8 @@ this example. cd ~/clearlinux/packages/dmidecode/ make build -#. The resulting RPMs are in :file:`./rpms`. Logs are in :file:`./results`. +#. The resulting RPMs are in :file:`./rpms`. Build logs and additional RPMs are + in :file:`./results`. Example 3: Build a new RPM ========================== @@ -187,54 +205,26 @@ this example. cd ~/clearlinux make autospecnew URL="https://github.com/clearlinux/helloclear/archive/helloclear-v1.0.tar.gz" NAME="helloclear" -#. The resulting RPMs are in :file:`./rpms`. Logs are in :file:`./results`. - -#. If build failures or dependency issues occur, provide the necessary - dependency, ban, or exclusion information via control files to autospec: - - #. Navigate to the specific package. - - .. code-block:: bash - - cd ~/clearlinux/packages/ - - #. Respond to the build process output by editing control files to resolve - issues, which may include dependencies or exclusions. See - `autospec README`_ for more information on control files. - - #. Run autospec again: - - .. code-block:: bash - - make autospec - - #. Repeat the last two steps above until all errors are resolved and you - complete a successful build. + The resulting RPMs are in :file:`./packages/helloclear/rpms`. Builde logs and + additional RPMs are in :file:`./packages/helloclear/results`. Example 4: Generate a new spec file with a pre-defined package ============================================================== This example shows how to modify an existing package to create a custom RPM. In -this example you will make a simple change to the ``dmidecode`` package, change -the revision to a new number higher than the |CL| OS version, and rebuild the -package. +this example you will make a simple change to the ``dmidecode`` package and +rebuild the package. Make sure you have completed `Example 1: First time setup`_ before starting this example. -#. Navigate to the autospec workspace and clone the ``dmidecode`` package. +#. Navigate to the autospec workspace and clone the ``dmidecode`` package: .. code-block:: bash cd ~/clearlinux make clone_dmidecode - Or you can clone all package repos at once using: - - .. code-block:: bash - - make [-j NUM] clone-packages - #. Navigate into the *dmidecode* directory: .. code-block:: bash @@ -265,6 +255,119 @@ this example. #. The resulting RPMs are in :file:`./rpms`. Logs are in :file:`./results`. +Example 5: Provide control files to autospec +============================================ + +This example shows how to modify control files to correct build failures that +autospec is unable to resolve. In this example you will add a missing license +and dependencies in order for autospec to complete a successful build. + +Make sure you have completed `Example 1: First time setup`_ before starting +this example. + +#. Navigate to the autospec workspace: + + .. code-block:: bash + + cd ~/clearlinux + +#. Build the opae-sdk RPM: + + .. code-block:: bash + + make autospecnew URL="https://github.com/OPAE/opae-sdk/archive/0.13.0.tar.gz" NAME="opae-sdk" + + This will give an error for a missing license file: + + .. code-block:: console + + [FATAL] Cannot find any license or opae-sdk.license file! + +#. Navigate to the package with build failures: + + .. code-block:: bash + + cd packages/opae-sdk + +#. Add a license: + + .. code-block:: bash + + echo "MIT" > opae-sdk.license + +#. Run autospec again: + + .. code-block:: bash + + make autospec + + This will result in a generic error: + + .. code-block:: console + + [FATAL] Build failed, aborting + +#. Open the build log to view the error details: + + .. code-block:: bash + + cat ./results/build.log + + In the build log, you will find details for the specific failures. In this + instance, there are missing dependencies: + + .. code-block:: console + + CMake Error: The following variables are used in this project, but they are set to NOTFOUND. Please set them or make sure they are set and tested correctly in the CMake files: + CJSON_LIBRARY + linked by target "opae-c++-utils" in directory /builddir/build/BUILD/opae-sdk-0.13.0/tools/c++utilslib + json-c_LIBRARIES + linked by target "opae-c" in directory /builddir/build/BUILD/opae-sdk-0.13.0/libopae + libuuid_LIBRARIES + linked by target "opae-c" in directory /builddir/build/BUILD/opae-sdk-0.13.0/libopae + +#. Search for the json-c package in |CL|: + + .. code-block:: bash + + grep 'json-c\.so$' ~/clearlinux/packages/*/*.spec + +#. Add the json-c-dev dependency: + + .. code-block:: bash + + echo "json-c-dev" >> buildreq_add + +#. Search for the libuuid package in |CL| and add the util-linux-dev dependency: + + .. code-block:: bash + + grep 'libuuid\.so$' ~/clearlinux/packages/*/*.spec + echo "util-linux-dev" >> buildreq_add + +#. Run autospec again and find the successfully-generated RPMs in the rpms + directory: + + .. code-block:: bash + + make autospec + +.. note:: + + If you need a dependency that does not exist in the |CL| repo, you must first + build it manually, then add the repo so that autospec knows the package + exists. For example: + + .. code-block:: bash + + cd ~/clearlinux/packages/ + make repoadd + make repostatus + + You only need to add the dependency to the :file:`buildreq_add` control file + if autospec is not able to automatically guess the correct dependency on its + own. + Related topics ************** @@ -274,5 +377,6 @@ Related topics .. _user-setup script: https://github.com/clearlinux/common/blob/master/user-setup.sh .. _autospec README: https://github.com/clearlinux/autospec .. _control files: https://github.com/clearlinux/autospec#control-files +.. _mock wiki: https://github.com/rpm-software-management/mock/wiki .. _rpm website: http://rpm.org .. _RPM Packaging Guide: https://rpm-packaging-guide.github.io/