From 310f01deebb1d08483d3bc23819a74d38d0f4299 Mon Sep 17 00:00:00 2001 From: "Kevin C. Wells" Date: Wed, 24 Jan 2018 16:34:10 -0800 Subject: [PATCH 1/8] Update Mixer documentation Updating the documentation for mixer to account for recent changes, and to better clarify some concepts. This is to coincide with an upcoming release of mixer, in which several important changes will be introduced. 1) The mixer CLI has been rewritten, supporting POSIX-style flags and a modern command layout. This is a breaking change, as commands and flags have been renamed. 2) The initialization steps for mixer have been further automated, so the documentation has been updated to account for the new, simpler approach. 3) Documentation has been added for the recent command for automating the addition of bundles to a mix. 4) The tool now handles caching upstream CLR bundle definitions in a more automated way, and so the 'mixer get-bundles' command (and its corresponding documentation section) has been removed. Signed-off-by: Kevin C. Wells --- .../clear-linux/guides/maintenance/mixer.rst | 288 ++++++++++-------- 1 file changed, 163 insertions(+), 125 deletions(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index be802d11..a50650a7 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -30,29 +30,42 @@ Mixing use as a "workspace" for mixing. For these steps, we assume your workspace location is :file:`/home/clr/mix`. -#. **Configure builder.conf**. Copy the template conf file +#. **Generate the starting point for your Mix**. In your workspace, run:: - .. code-block:: console + # sudo mixer init --clear-version 13180 --mix-version 10 - # cp /usr/share/defaults/bundle-chroot-builder/builder.conf /home/clr/mix/ + This will initialize your workspace so that you can make a mix at version 10 + based on upstream Clear Linux version 13180. A default :file:`builder.conf` file + will be created (if one doesn't already exist in your workspace), and a + :file:`mix-bundles` directory will be created. - The file ``builder.conf`` will be read automatically from the current - workspace directory, but the ``-config`` option exists to specify where the - file is if you want to store it elsewhere. To use one in your current - workspace, copy the template to /home/clr/mix. The :file:`.yum-mix.conf` - file will be auto-generated for you, as will the :file:`Swupd_Root.pem`. A - yum configuration is needed for the chroot-builder to know where the RPMs - are hosted, and the certificate file is needed to sign the root Manifest to - provide security for content verification. + If you intend to build a mix with your own custom RPMs, run:: - Note there are different sections to the builder.conf. The ``[Builder]`` - section provides the mixer tools with required configuration options, + # sudo mixer init --clear-version 13180 --mix-version 10 --local-rpms + + This will create :file:`local` and :file:`rpms` directories in your mix, and + add their paths to the generated :file:`builder.conf`. (For more information + on using these directories, or setting them up manually, see Step 4 below.) + + If you know you want to build a mix that includes all Clear bundles with no + modifications, you can easily do so by running:: + + # sudo mixer init --clear-version 13180 --mix-version 10 --all + +#. **Configure builder.conf**. Edit the :file:`builder.conf` as needed. + + The :file:`builder.conf` file will be read automatically from the current + workspace directory, but the ``--config`` option exists to specify where the + file is if you want to store it elsewhere. + + Note there are different sections to the :file:`builder.conf`. The ``[Builder]`` + section provides the mixer tools with the required configuration options, defining where generated bundles and update metadata should get published. The ``[swupd]`` section is used by swupd-server to create an update with specific update parameters. Edit the template configuration file according to your needs. For this - example, your ``builder.conf`` should look like this, with both URL + example, your :file:`builder.conf` should look like this, with both URL variables set to the domain or IP of the update server:: # vim /etc/bundle-chroot-builder/builder.conf: @@ -70,195 +83,220 @@ Mixing VERSIONURL= FORMAT=1 - The SERVER_STATE_DIR is where the mix content will be outputted to, and it + The ``SERVER_STATE_DIR`` is where the mix content will be output, and it is automatically created for you by the mixer. This can be set to any location, but for this example let's use the workspace directory. The same - applies for BUNDLE_DIR; it will be generated for you in the location - specified in the builder.conf, in this case ``/home/clr/mix/mix-bundles``. - This is where the bundle definitions are stored for your mix, and it's where - the chroot-builder looks to know what bundles must be installed. + applies for ``BUNDLE_DIR``; it will be generated for you in the location + specified in the :file:`builder.conf`, in this case + ``/home/clr/mix/mix-bundles``. This is where the bundle definitions are + stored for your mix, and it's where the chroot-builder looks to know what + bundles must be installed. - You may change the ``CERT=/path/to/cert`` line, which tells the chroot - builder to insert the certificate specified for the mix in ``/os-core- - update/usr/share/clear/update-ca/``. This is the certificate used by the - software update client to verify the Manifest.MoM signature. For now, it is - `HIGHLY` recommended that you do not modify this line, as the certificate - swupd expects needs a very specific configuration to sign and verify - properly. The certificate will be automatically generated for you, and the - Manifest.MoM will be signed automatically as well, providing security for - the update content you create. + The :file:`.yum-mix.conf` file defined in ``YUM_CONF`` will be auto-generated + for you, as will the ``CERT`` file, :file:`Swupd_Root.pem`. A yum + configuration is needed for the chroot-builder to know where the RPMs are + hosted, and the certificate file is needed to sign the root Manifest to + provide security for content verification. - The CONTENTURL and VERSIONURL may be an IP address, or a domain name, which - hosts the /home/clr/mix/update/www (SERVER_STATE_DIR) directory. Creating a - symlink to the directory in your server webdir is an easy way to host the - content. A client running the mix will look to that URL to figure out if - there is a new version available and the location from which to download the - update content. + You may change the ``CERT=/path/to/cert`` line to point to a different + certificate. The chroot builder will insert the certificate specified here + in ``/os-core-update/usr/share/clear/update-ca/``. This is the certificate + used by the software update client to verify the :file:`Manifest.MoM` + signature. For now, it is **HIGHLY** recommended that you do not modify this + line, as the certificate swupd expects needs a very specific configuration to + sign and verify properly. The certificate will be automatically generated for + you, and the Manifest.MoM will be signed automatically as well, providing + security for the update content you create. - To learn more about the FORMAT option, please refer to the bottom of this - document "Format Version" and https://github.com/clearlinux/swupd-server/wiki/Format-Bumps. - For now leave the FORMAT value alone and do not increment it. + The ``CONTENTURL`` and ``VERSIONURL`` should be set to the domain or IP + address where your update content will be served. This is the location that + hosts the :file:`/home/clr/mix/update/www` (``SERVER_STATE_DIR``) directory. + Creating a symlink to the directory in your server webdir is an easy way to + host the content. These URLs are embeded in images created for your mix. They + are where ``swupd-client`` will look to figure out if there is a new version + available, and the location from which to download the update content. Think + of these as the equivalent of https://cdn.download.clearlinux.org/update/ + used by Clear Linux, but for your derivative mix. + + To learn more about the ``FORMAT`` option, please refer to the "Format + Version" section at the bottom of this document, and + https://github.com/clearlinux/swupd-server/wiki/Format-Bumps. For now, leave + the ``FORMAT`` value alone and do not increment it. The mix version and Clear version will come from two state files: :file:`.mixversion` and :file:`.clearversion`, both of which will be created - for you when you set-up the workspace and added to the VERSIONS_PATH - defined. - -#. **Generate the starting point for your Mix**. In your workspace, run:: - - # sudo mixer init-mix -clearver 13180 -mixver 10 - - *If you wish to just build a mix that includes all Clear bundles with no modifications, run*:: - - # sudo mixer init-mix -all -clearver 13180 -mixver 10 + for you when you set-up the workspace. They will be created in the directory + defined by the ``VERSIONS_PATH``. #. **Create/locate RPMs for mix.**. (Steps 4-6 are necessary only if you want to add your own RPMs to the Mix. If you are working only with Clear bundles, then skip to Step 7.) If you are creating RPMs from scratch, you may use ``autospec``, - ``mock``, ``rpmbuild``, etc. to build them. If they are not - built on Clear, make sure your configuration and toolchain builds them correctly for Clear, or there is no guarantee - they will be compatible. + ``mock``, ``rpmbuild``, etc. to build them. If they are not built on Clear, + make sure your configuration and toolchain builds them correctly for Clear, + or there is no guarantee they will be compatible. -#. **Import RPMs into workspace**. The way to do this is to create an - ``rpms`` directory in your workspace (for example ``/home/clr/mix/rpms``), - and to copy the RPMs you want into that directory. The mixer script will - look here for RPMs in order to build a local RPM repo for yum to use. +#. **Import RPMs into workspace**. Create an :file:`rpms` directory in your + workspace (for example :file:`/home/clr/mix/rpms`), and copy the RPMs you + want into that directory. Next, add the following to your + :file:`builder.conf`:: + + RPMDIR=/home/clr/mix/rpms + + Mixer will look here for RPMs in order to build a local RPM repo for yum to + use. #. **Create a local RPM repo**. Create an empty directory in your workspace - named ``local`` and add the paths in your builder.conf:: + named :file:`local` and add the path in your :file:`builder.conf`:: - RPMDIR=/home/clr/mix/rpms - REPODIR=/home/clr/mix/local + REPODIR=/home/clr/mix/local - These variables are automatically read; you simply need to run:: + Once these values are configured, you can generate the yum repo by running:: # sudo mixer add-rpms - After the script exits, you should see your RPMs and a repodata directory in - ``/home/clr/mix/local``. If the RPMs are not all in the local directory, check - to make sure that they are indeed valid RPM files and not corrupt. + After the tool exits, you should see your RPMs and a repodata directory in + :file:`/home/clr/mix/local`. If the RPMs are not all in this :file:`local` + directory, check to make sure that they are indeed valid RPM files and not + corrupt. -#. **Update/Add bundle definitions**. The mixer uses a local clone of the - ``clr-bundles`` repo to define bundles for the mix. +#. **Update/Add bundle definitions**. You can easily add bundles to your mix by + running:: + + # sudo mixer bundle add bundle1,bundle2,... + + This command will copy the specified bundle defintion files from your + configured upstream version of Clear Linux (:file:`.clearversion`) into your + :file:`mix-bundles` directory. + + Behind the scenes, mixer uses a local cache of the upstream Clear Linux + bundle definitions. These are stored in the + :file:`.mixer/upstream-bundles/clr-bundles-{VER}/bundles/` directory in your + workspace. Do *not* modify things in this directory; it is simply a mirror + for the tool to use. However, you can refer to the files in this directory + to see what bundles are available, or the format these files should have. To define your bundles: - #. Navigate to the ``mix-bundles/`` directory. + #. Navigate to the :file:`mix-bundles/` directory. #. Make any needed modifications to the bundle set. #. Commit the result:: $ git add . $ git commit -s -m 'Update bundles for mix #' - You can easily copy bundles over from the - ``clr-bundles/clr-bundles-VER/bundles/`` directory in the case that you - want to simply use existing bundle sets. Note that ``mix-bundles`` should - not have any folders inside of it, only bundle definitions. - Do *not* modify things in the clr-bundles dir, this is simply a mirror for - you to use or refer to the Clear Linux OS bundle definitions. - - Why do this? With Git history, mixes are easy to revert to or refer - to in the future if something were to go wrong with a new mix. If + While using Git is optional, with Git history, mixes are easy to revert to or + refer to in the future if something were to go wrong with a new mix. If you're just testing this out, or if you really do not want to mess with Git, you can ignore committing for now. - To add your own bundle, create a bundle definition file in ``mix-bundles/`` - and refer to :file:`mix-bundles/os-core-update` for formatting, but be sure - that the name does not conflict with another bundle. Add your package - name(s) in that bundle definition file to tell it what package(s) must be - installed as part of that bundle. + To add your own bundle, create a bundle definition file in the correct format + in the :file:`mix-bundles` directory (you can refer to an existing bundle, + like :file:`mix-bundles/os-core-update`, for formatting). Be sure that the + bundle name you choose does not conflict with another bundle. Add your + package name(s) in the bundle definition file to tell it what package(s) + must be installed as part of that bundle. #. **Build the bundle chroots** To build all of the ``chroots`` that are based on the bundles you defined, in your workspace run:: - # sudo mixer build-chroots + # sudo mixer build chroots If you have many bundles defined for your mix, this step may take some time. #. **Create update**. In the workspace, run:: - # sudo mixer build-update + # sudo mixer build update When the build completes, you'll find your mix update content under - ``/home/clr/mix/update/www/VER``. In this example, it will be located in - ``/home/clr/mix/update/www/``, where is the mix - version you defined, or 10 by default. + :file:`/home/clr/mix/update/www/VER`. In this example, it will be located in + :file:`/home/clr/mix/update/www/{}`, where ```` is + the mix version you defined, or 10 by default. - All content to make a fully usable mix will be created by this step, but - note that only zero packs are automatically generated. To create optional - delta packs, run the pack-maker as follows:: + All content to make a fully usable mix will be created by this step, but note + that only "zero packs" are automatically generated. Zero packs are the + content needed to go from nothing to the mix version you just built content + for. To create optional "delta packs", which allow for transitioning from + one mix version to another, run the pack-maker as follows:: # sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update The pack-maker will generate all delta packs for changed bundles from - PAST_VERSION to MIX_VERSION. If your STATE_DIR is in a different location be - sure to specify where with the -S option. For the first build, no delta - packs can be created because the "update" is from version 0, which impicitly - has no content, thus no deltas can be generated. For subsequent builds, - mixer-pack-maker.sh can be run to generate delta content between them (i.e + ``PAST_VERSION`` to ``MIX_VERSION``. If your ``STATE_DIR`` is in a different + location, be sure to specify where with the ``-S`` option. For the first + build, no delta packs can be created because the "update" is from version 0, + which impicitly has no content, thus no deltas can be generated. For + subsequent builds, :file:`mixer-pack-maker.sh` can be run to generate delta + content between them (i.e 10 to 20). -#. **Creating an image** To create a bootable image from your update content, - you will need the configuration file for ister to create images:: +#. **Creating an image** Mixer uses the ``ister`` tool to create a bootable + image from your update content. To configure the image ``ister`` creates, + you'll need the ``ister`` config file. A default value can be obtained from + the ``ister`` package:: - # curl -O https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json + # cp /usr/share/defaults/ister/ister.json relase-image-config.json - Edit this to include all the bundles you want pre-installed into your - image. For a minimal, base image this would be:: + Feel free to inspect the config Clear Linux uses for its releases, which can + be found here: https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json + + Note that mixer will automatically look for a file named ``release-image- + config.json``, but you can choose whatever name you want. To use a different + name, simply pass the ``--template path/to/file.config`` flag when creating + your image. + + Edit the config to include all the bundles you want **pre-installed** into + your image. The rest of the bundles in your mix will be available to your + users via ``swupd bundle add``. Keeping this list small allows for a smaller + image size. For a minimal, base image this would be:: "Bundles": ["os-core", "os-core-update", "kernel-native"] - And lastly, set the "Version:" to say which mix version content the image - should be built from, i.e. 10 for your first build. To build the image, - run:: + Next, set the "Version" field to say which mix version content the image + should be built from. ``ister`` allows you to build an image from any mix + version you've built, not just the current one. For the first build example + we've been using, "Version" would be set to 10. - # sudo mixer build-image -format 1 + Finally, to build the image, run:: + + # sudo mixer build image --format 1 The output from this should be an image that is bootable as a VM or installable to baremetal. .. note:: - You need to pass in -format if the format you are - building is different than the format of Clear Linux OS you are currently - building on. Format version can be found via + By default, ``ister`` uses the format version of the build machine it is + running on. As such, you need to pass in ``--format `` if + the format you are building is different than the format of Clear Linux OS + you are currently building on. Your current format version can be found by + running - .. code-block:: console + .. code-block:: console - # cat /usr/share/defaults/swupd/format + # cat /usr/share/defaults/swupd/format Creating your next Mix version ------------------------------ -#. **Initialize next Mix version info**. To update the versions and prep for - your next mix: +**Update the next Mix version info**. Update the :file:`.mixversion` file to the +next version number you want to build. - Update the .mixversion file to the next version number you want to build. - From this point you can iterate through, starting again at step 4 and doing - modifications as needed. For example: +From this point you can iterate through, starting again at step 4 and doing +modifications as needed. For example: - - Add/Remove/Modify Bundles - - sudo mixer build-chroots - - sudo mixer build-update - - (Optionally) sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update +- Add/Remove/Modify Bundles +- ``sudo mixer build chroots`` +- ``sudo mixer build update`` +- (Optionally) ``sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update`` -#. **Update Bundles (Optional)**. Update ``clr-bundles``. In the workspace, - run:: - - # sudo mixer get-bundles - - This step is optional because it is only needed when you want to update the - upstream clr-bundles in your workspace to a new version, which requires - updating the .clearversion file. - Format Version -------------- -The "format" used in ``builder.conf`` might be more precisely referred to as an -OS "compatibility epoch". Versions of the OS within a given epoch are fully +The "format" used in :file:`builder.conf` might be more precisely referred to as +an OS "compatibility epoch". Versions of the OS within a given epoch are fully compatible with themselves and can update to any version in that epoch. Across the format boundary *something* has changed in the OS, such that updating from build M in format X, to build N in format Y will not work. Generally this From 5b4690f9d32a8be07149bd549d9aa7a744829c95 Mon Sep 17 00:00:00 2001 From: Ecouzens Date: Tue, 6 Feb 2018 16:30:39 -0800 Subject: [PATCH 2/8] Formatting and copy edit of mixer maintenance guide Signed-off-by: Ecouzens --- .../clear-linux/guides/maintenance/mixer.rst | 406 ++++++++++-------- 1 file changed, 223 insertions(+), 183 deletions(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index a50650a7..69178ad4 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -3,70 +3,79 @@ Use mixer tool ############## -*Mixing* refers to composing an operating system for specific use cases. While -the default Clear Linux\* OS for Intel® Architecture provides options to -install bundles for various server capabilities, some developers may wish to 1) -augment the operating system itself with functionality from their own packages -or 2) modify the structure of current bundles to cater to their particular -needs. +*Mixing* refers to composing an operating system for specific use cases. +While the default Clear Linux* OS for Intel® Architecture provides options +to install bundles for various server capabilities, some developers may wish +to either augment the operating system itself with functionality from their +own packages, or modify the structure of current bundles to cater to their +particular needs. Prerequisites -============= +************* -To start working with the Mixer tools, you'll need a recent image of Clear +To start working with the mixer tools, you'll need a recent image of Clear Linux OS for Intel Architecture with the following bundle installed. If you don't have it already, you can add it with the :command:`swupd bundle-add` -command like so:: +command like so: - # swupd bundle-add mixer + .. code-block:: bash -Current Workflow -================ + # swupd bundle-add mixer -Mixing ------- +Current mixing workflow +*********************** #. **Create a workspace**. Create an empty directory in your Clear image to use as a "workspace" for mixing. For these steps, we assume your workspace location is :file:`/home/clr/mix`. -#. **Generate the starting point for your Mix**. In your workspace, run:: +#. **Generate the starting point for your mix**. In your workspace, run: + + .. code-block:: bash # sudo mixer init --clear-version 13180 --mix-version 10 - This will initialize your workspace so that you can make a mix at version 10 - based on upstream Clear Linux version 13180. A default :file:`builder.conf` file - will be created (if one doesn't already exist in your workspace), and a - :file:`mix-bundles` directory will be created. + This initializes your workspace so that you can make a mix at version 10 + based on upstream Clear Linux version 13180. A default :file:`builder.conf` + file will be created (if one is not already present in your workspace) + along with a :file:`mix-bundles` directory. - If you intend to build a mix with your own custom RPMs, run:: + If you intend to build a mix with your own custom RPMs, run: + + .. code-block:: bash # sudo mixer init --clear-version 13180 --mix-version 10 --local-rpms - This will create :file:`local` and :file:`rpms` directories in your mix, and - add their paths to the generated :file:`builder.conf`. (For more information - on using these directories, or setting them up manually, see Step 4 below.) + This creates :file:`local` and :file:`rpms` directories in your mix, and + adds their paths to the generated :file:`builder.conf`. (For more + information on using these directories, or setting them up manually, see + Step 4 below.) - If you know you want to build a mix that includes all Clear bundles with no - modifications, you can easily do so by running:: + You can easily build a mix that includes all Clear bundles with no + modifications using the following command: + + .. code-block:: bash # sudo mixer init --clear-version 13180 --mix-version 10 --all #. **Configure builder.conf**. Edit the :file:`builder.conf` as needed. The :file:`builder.conf` file will be read automatically from the current - workspace directory, but the ``--config`` option exists to specify where the - file is if you want to store it elsewhere. + workspace directory, but the :option:`--config` option exists to specify + where the file is if you want to store it elsewhere. - Note there are different sections to the :file:`builder.conf`. The ``[Builder]`` - section provides the mixer tools with the required configuration options, - defining where generated bundles and update metadata should get published. - The ``[swupd]`` section is used by swupd-server to create an update with - specific update parameters. + Note there are different sections of :file:`builder.conf`. The + ``[Builder]`` section provides the mixer tools with the required + configuration options, and defines where generated bundles and updated + metadata should be published. The ``[swupd]`` section is used by + swupd-server to create an update with specific update parameters. Edit the template configuration file according to your needs. For this - example, your :file:`builder.conf` should look like this, with both URL - variables set to the domain or IP of the update server:: + example, your :file:`builder.conf` should look similar to the example + below, with both URL variables set to the domain or IP of the update + server: + + .. code-block:: console # vim /etc/bundle-chroot-builder/builder.conf: @@ -83,76 +92,83 @@ Mixing VERSIONURL= FORMAT=1 - The ``SERVER_STATE_DIR`` is where the mix content will be output, and it - is automatically created for you by the mixer. This can be set to any - location, but for this example let's use the workspace directory. The same - applies for ``BUNDLE_DIR``; it will be generated for you in the location - specified in the :file:`builder.conf`, in this case - ``/home/clr/mix/mix-bundles``. This is where the bundle definitions are - stored for your mix, and it's where the chroot-builder looks to know what - bundles must be installed. + The ``SERVER_STATE_DIR`` is where the mixed content is output. This + is automatically created for you by the mixer. You can set this + directory to any location, but we will use the workspace directory for + this example. The same applies for ``BUNDLE_DIR``. This directory is + generated for you in the location specified in the :file:`builder.conf`, + in this case ``/home/clr/mix/mix-bundles``. This is where the bundle + definitions are stored for your mix, and it is where the chroot-builder + looks to know what bundles must be installed. - The :file:`.yum-mix.conf` file defined in ``YUM_CONF`` will be auto-generated - for you, as will the ``CERT`` file, :file:`Swupd_Root.pem`. A yum - configuration is needed for the chroot-builder to know where the RPMs are - hosted, and the certificate file is needed to sign the root Manifest to - provide security for content verification. + The :file:`.yum-mix.conf` file defined in ``YUM_CONF`` is auto-generated, + along with the ``CERT`` file, :file:`Swupd_Root.pem`. The yum configuration + file is needed for the chroot-builder to know where the RPMs are hosted, + and the certificate file is needed to sign the root Manifest to provide + security for content verification. - You may change the ``CERT=/path/to/cert`` line to point to a different - certificate. The chroot builder will insert the certificate specified here + You can change the ``CERT=/path/to/cert`` line to point to a different + certificate. The chroot builder inserts the certificate specified here in ``/os-core-update/usr/share/clear/update-ca/``. This is the certificate used by the software update client to verify the :file:`Manifest.MoM` - signature. For now, it is **HIGHLY** recommended that you do not modify this - line, as the certificate swupd expects needs a very specific configuration to - sign and verify properly. The certificate will be automatically generated for - you, and the Manifest.MoM will be signed automatically as well, providing - security for the update content you create. + signature. For now, we *highly* recommend that you do not modify this + line, as the certificate that swupd expects needs a very specific + configuration to sign and verify properly. The certificate is + automatically generated, and the :file:`Manifest.MoM` is signed + automatically as well, providing security for the updated content that + you create. The ``CONTENTURL`` and ``VERSIONURL`` should be set to the domain or IP - address where your update content will be served. This is the location that - hosts the :file:`/home/clr/mix/update/www` (``SERVER_STATE_DIR``) directory. - Creating a symlink to the directory in your server webdir is an easy way to - host the content. These URLs are embeded in images created for your mix. They - are where ``swupd-client`` will look to figure out if there is a new version - available, and the location from which to download the update content. Think - of these as the equivalent of https://cdn.download.clearlinux.org/update/ - used by Clear Linux, but for your derivative mix. + address where your updated content will be served. This is the location + that hosts the :file:`/home/clr/mix/update/www` (``SERVER_STATE_DIR``) + directory. Creating a symlink to the directory in your server webdir is + an easy way to host the content. These URLs are embeded in images created + for your mix. They are where ``swupd-client`` will look to figure out if + there is a new version available, and the location from which to download + the updated content. Think of these as the equivalent of the `Clear Linux update page`_ used by Clear Linux, but for your derivative mix. - To learn more about the ``FORMAT`` option, please refer to the "Format - Version" section at the bottom of this document, and - https://github.com/clearlinux/swupd-server/wiki/Format-Bumps. For now, leave - the ``FORMAT`` value alone and do not increment it. + To learn more about the ``FORMAT`` option, refer to the "Format Version" + section at the bottom of this document, and `Format Bumps`_ on the Clear + Linux wiki. For now, leave the ``FORMAT`` value alone and do not increment + it. The mix version and Clear version will come from two state files: - :file:`.mixversion` and :file:`.clearversion`, both of which will be created - for you when you set-up the workspace. They will be created in the directory - defined by the ``VERSIONS_PATH``. + :file:`.mixversion` and :file:`.clearversion`, both of which will be + created for you when you set up the workspace. They will be created in the + directory defined by the ``VERSIONS_PATH``. -#. **Create/locate RPMs for mix.**. (Steps 4-6 are necessary only if you - want to add your own RPMs to the Mix. If you are working only with Clear - bundles, then skip to Step 7.) +.. _step-four: - If you are creating RPMs from scratch, you may use ``autospec``, - ``mock``, ``rpmbuild``, etc. to build them. If they are not built on Clear, - make sure your configuration and toolchain builds them correctly for Clear, - or there is no guarantee they will be compatible. +#. **Create/locate RPMs for mix.**. (Steps 4 through 6 are necessary only + if you want to add your own RPMs to the Mix. If you are working only with Clear bundles, then skip to Step 7.) + + If you are creating RPMs from scratch, you can use ``autospec``, ``mock``, + ``rpmbuild``, etc. to build them. If they are not built on Clear, + make sure your configuration and toolchain builds them correctly for Clear, or there is no guarantee they will be compatible. #. **Import RPMs into workspace**. Create an :file:`rpms` directory in your workspace (for example :file:`/home/clr/mix/rpms`), and copy the RPMs you want into that directory. Next, add the following to your - :file:`builder.conf`:: + :file:`builder.conf`: + + .. code-block:: bash RPMDIR=/home/clr/mix/rpms - Mixer will look here for RPMs in order to build a local RPM repo for yum to - use. + Mixer will look in this directory for RPMs to build a local RPM repo for + yum to use. #. **Create a local RPM repo**. Create an empty directory in your workspace - named :file:`local` and add the path in your :file:`builder.conf`:: + named :file:`local` and add the path in your :file:`builder.conf`: + + .. code-block:: bash REPODIR=/home/clr/mix/local - Once these values are configured, you can generate the yum repo by running:: + Once these values are configured, you can generate the yum repo by + running the following command: + + .. code-block:: bash # sudo mixer add-rpms @@ -161,158 +177,182 @@ Mixing directory, check to make sure that they are indeed valid RPM files and not corrupt. -#. **Update/Add bundle definitions**. You can easily add bundles to your mix by - running:: +#. **Update/Add bundle definitions**. You can easily add bundles to your mix + by running: + + .. code-block:: bash # sudo mixer bundle add bundle1,bundle2,... - This command will copy the specified bundle defintion files from your - configured upstream version of Clear Linux (:file:`.clearversion`) into your - :file:`mix-bundles` directory. + This command copies the specified bundle defintion files from your + configured upstream version of Clear Linux (:file:`.clearversion`) into + your :file:`mix-bundles` directory. Behind the scenes, mixer uses a local cache of the upstream Clear Linux bundle definitions. These are stored in the - :file:`.mixer/upstream-bundles/clr-bundles-{VER}/bundles/` directory in your - workspace. Do *not* modify things in this directory; it is simply a mirror - for the tool to use. However, you can refer to the files in this directory - to see what bundles are available, or the format these files should have. + :file:`.mixer/upstream-bundles/clr-bundles-{VER}/bundles/` directory in + your workspace. Do *not* modify things in this directory; it is simply a + mirror for the tool to use. However, you can refer to the files in this directory to see what bundles are available, or the format these files should have. To define your bundles: #. Navigate to the :file:`mix-bundles/` directory. #. Make any needed modifications to the bundle set. - #. Commit the result:: + #. Commit the result: + + .. code-block:: bash $ git add . $ git commit -s -m 'Update bundles for mix #' + While using Git is optional, with Git history, mixes are easy to revert + or refer to in the future if something goes wrong with a new mix. If + you're just testing this out, or if you really do not want to mess with + Git, you can ignore committing for now. - While using Git is optional, with Git history, mixes are easy to revert to or - refer to in the future if something were to go wrong with a new mix. If - you're just testing this out, or if you really do not want to mess with Git, - you can ignore committing for now. + To add your own bundle, create a bundle definition file in the correct + format in the :file:`mix-bundles` directory (you can refer to an existing + bundle, like :file:`mix-bundles/os-core-update`, for formatting). Be sure that the bundle name you choose does not conflict with another bundle. + Add your package name(s) in the bundle definition file to tell it what package(s) must be installed as part of that bundle. - To add your own bundle, create a bundle definition file in the correct format - in the :file:`mix-bundles` directory (you can refer to an existing bundle, - like :file:`mix-bundles/os-core-update`, for formatting). Be sure that the - bundle name you choose does not conflict with another bundle. Add your - package name(s) in the bundle definition file to tell it what package(s) - must be installed as part of that bundle. +#. **Build the bundle chroots**. To build all of the ``chroots`` that are + based on the bundles you defined, run the following command in your + workspace: -#. **Build the bundle chroots** To build all of the ``chroots`` - that are based on the bundles you defined, in your workspace run:: + .. code-block:: bash # sudo mixer build chroots - If you have many bundles defined for your mix, this step may take some time. + If you have many bundles defined for your mix, this step might take some + time. -#. **Create update**. In the workspace, run:: +#. **Create update**. In the workspace, run: + + .. code-block:: bash # sudo mixer build update - When the build completes, you'll find your mix update content under - :file:`/home/clr/mix/update/www/VER`. In this example, it will be located in - :file:`/home/clr/mix/update/www/{}`, where ```` is - the mix version you defined, or 10 by default. + When the build completes, you will find your mix update content under + :file:`/home/clr/mix/update/www/VER`. In our example, this will be + located in :file:`/home/clr/mix/update/www/{}`, where + ```` is the mix version you defined (10 by default). - All content to make a fully usable mix will be created by this step, but note - that only "zero packs" are automatically generated. Zero packs are the - content needed to go from nothing to the mix version you just built content - for. To create optional "delta packs", which allow for transitioning from - one mix version to another, run the pack-maker as follows:: + All content to make a fully usable mix will be created by this step, but + note that only *zero packs* are automatically generated. Zero packs are + the content needed to go from nothing to the mix version you just built + content for. To create optional *delta packs*, which allow for + transitioning from one mix version to another, run the pack-maker as + follows: + + .. code-block:: bash # sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update - The pack-maker will generate all delta packs for changed bundles from - ``PAST_VERSION`` to ``MIX_VERSION``. If your ``STATE_DIR`` is in a different - location, be sure to specify where with the ``-S`` option. For the first - build, no delta packs can be created because the "update" is from version 0, - which impicitly has no content, thus no deltas can be generated. For - subsequent builds, :file:`mixer-pack-maker.sh` can be run to generate delta - content between them (i.e - 10 to 20). + The pack-maker will generate all delta packs for bundles that have changed + from ``PAST_VERSION`` to ``MIX_VERSION``. If your ``STATE_DIR`` is in a + different location, be sure to specify the location with the ``-S`` + option. For the first build, no delta packs can be created because the + "update" is from version 0. Version 0 impicitly has no content, thus no + deltas can be generated. For subsequent builds, + :file:`mixer-pack-maker.sh` can be run to generate delta content between + them (for example: 10 to 20). -#. **Creating an image** Mixer uses the ``ister`` tool to create a bootable - image from your update content. To configure the image ``ister`` creates, - you'll need the ``ister`` config file. A default value can be obtained from - the ``ister`` package:: +#. **Creating an image**. Mixer uses the ``ister`` tool to create a bootable + image from your updated content. To configure the image ``ister`` creates, + you will need the ``ister`` config file. You can obtain a default value + from the ``ister`` package: + + .. code-block:: bash # cp /usr/share/defaults/ister/ister.json relase-image-config.json - Feel free to inspect the config Clear Linux uses for its releases, which can - be found here: https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json + For reference, you can inspect the ``ister`` config file that `Clear + Linux uses`_ for its releases. - Note that mixer will automatically look for a file named ``release-image- - config.json``, but you can choose whatever name you want. To use a different - name, simply pass the ``--template path/to/file.config`` flag when creating - your image. + Note that mixer automatically looks for a file named :file:`release-image- + config.json`, but you can choose whatever name you want. To use a + different name, simply pass the :option:`--template path/to/file.config` + flag when creating your image. - Edit the config to include all the bundles you want **pre-installed** into - your image. The rest of the bundles in your mix will be available to your - users via ``swupd bundle add``. Keeping this list small allows for a smaller - image size. For a minimal, base image this would be:: + Edit the config file to include all bundles that you want *preinstalled* + into your image. The rest of the bundles in your mix will be available to your users via: - "Bundles": ["os-core", "os-core-update", "kernel-native"] + .. code-block:: bash - Next, set the "Version" field to say which mix version content the image - should be built from. ``ister`` allows you to build an image from any mix - version you've built, not just the current one. For the first build example - we've been using, "Version" would be set to 10. + # swupd bundle add - Finally, to build the image, run:: - - # sudo mixer build image --format 1 - - The output from this should be an image that is bootable as a VM or - installable to baremetal. - - .. note:: - By default, ``ister`` uses the format version of the build machine it is - running on. As such, you need to pass in ``--format `` if - the format you are building is different than the format of Clear Linux OS - you are currently building on. Your current format version can be found by - running + Keeping this list small allows for a smaller image size. For a minimal, + base image, this list would be: .. code-block:: console + "Bundles": ["os-core", "os-core-update", "kernel-native"] + + Next, set the ``Version`` field to the mix version content that the image + should be built from. ``ister`` allows you to build an image from any mix + version that you have built, not just the current one. For the first build + example we've been using, ``Version`` would be set to 10. + + Finally, to build the image, run: + + .. code-block:: bash + + # sudo mixer build image --format 1 + + The output from this should be an image that is bootable as a virtual + machine and can be installed on bare metal. + + .. note:: + By default, ``ister`` uses the format version of the build machine it + is running on. Therefore, if the format you are building is different + than the format of the Clear Linux OS that you are building on, you + need to pass :option:`--format `. You can find your + current format version by running: + + .. code-block:: bash + # cat /usr/share/defaults/swupd/format -Creating your next Mix version ------------------------------- +Creating your next mix version +============================== -**Update the next Mix version info**. Update the :file:`.mixversion` file to the -next version number you want to build. +**Update the next Mix version info**. Update the :file:`.mixversion` file to +the next version number you want to build. -From this point you can iterate through, starting again at step 4 and doing -modifications as needed. For example: +From this point you can iterate through the instructions , starting again at +:ref:`step 4 ` and making modifications as needed. For example: -- Add/Remove/Modify Bundles -- ``sudo mixer build chroots`` -- ``sudo mixer build update`` -- (Optionally) ``sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update`` + - Add/remove/modify bundles + - ``sudo mixer build chroots`` + - ``sudo mixer build update`` + - (Optionally) ``sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update`` Format Version --------------- +************** -The "format" used in :file:`builder.conf` might be more precisely referred to as -an OS "compatibility epoch". Versions of the OS within a given epoch are fully -compatible with themselves and can update to any version in that epoch. Across -the format boundary *something* has changed in the OS, such that updating from -build M in format X, to build N in format Y will not work. Generally this -occurs when the software updater or manifests changed in a way that is no -longer compatible with the previous update scheme. +The ``format`` used in :file:`builder.conf` might be more precisely referred +to as an OS "compatibility epoch". Versions of the OS within a given epoch +are fully compatible with themselves and can update to any version in that +epoch. Across the ``format`` boundary *something* has changed in the OS, +such that updating from build M in format X, to build N in format Y will not +work. Generally this occurs when the software updater or manifests changed +in a way that is no longer compatible with the previous update scheme. -A format increment is the way we insure pre- and co-requisite changes flow out -with proper ordering. The update client will only ever update to the latest -release in its respective format version (unless overridden by command line -flags), thus we can guarantee all clients will update to the final version in -their given format, which *must* contain all the changes needed to understand -the content built in the following format. Only after reaching the final -release in the old format will a client be able to continue to update to -releases in the new format. +A format increment is the way we insure pre- and co-requisite changes flow +out with proper ordering. The updated client will only ever update to the +latest release in its respective format version (unless overridden by +command line flags). Thus we can guarantee all clients will update to the +final version in their given format, which *must* contain all the changes +needed to understand the content built in the subsequent format. Only after +reaching the final release in the old format will a client be able to +continue to update to releases in the new format. -For the creation of a custom mix, the format version should start at '1', or -some known number, and increment only when a compatibility breakage is -introduced. Normal updates (updating a software package for example) do not +When creating a custom mix, the format version should start at '1', or +some known number, and should increment only when a compatibility breakage is +introduced. Normal updates (for example, updating a software package) do not require a format increment. + +.. _Clear Linux update page: https://cdn.download.clearlinux.org/update/ +.. _Format Bumps: https://github.com/clearlinux/swupd-server/wiki/Format-Bumps +.. _Clear Linux uses: https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json \ No newline at end of file From 19995d3c2c904c22a66cef406e9af2234512c603 Mon Sep 17 00:00:00 2001 From: "Kevin C. Wells" Date: Tue, 27 Feb 2018 16:14:07 -0800 Subject: [PATCH 3/8] Update mixer documentation for new commands This patch updates the documentation to comply with all of the new changes Mixer 4.0 introduces: - Many changes to the bundle lifecycle. It introduces the concept of "local" and "upstream" bundles, and several new commands to work with them. - The new, native implementations of swupd-server and the bundle-chroot-builder tool, both still in experimental mode behind optional flags. - New default values for mixer init Signed-off-by: Kevin C. Wells --- .../clear-linux/guides/maintenance/mixer.rst | 373 ++++++++++++++---- 1 file changed, 297 insertions(+), 76 deletions(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index 69178ad4..d03f3b4b 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -8,7 +8,7 @@ While the default Clear Linux* OS for Intel® Architecture provides options to install bundles for various server capabilities, some developers may wish to either augment the operating system itself with functionality from their own packages, or modify the structure of current bundles to cater to their -particular needs. +particular needs. Prerequisites ************* @@ -33,30 +33,60 @@ Current mixing workflow .. code-block:: bash - # sudo mixer init --clear-version 13180 --mix-version 10 + # mixer init This initializes your workspace so that you can make a mix at version 10 - based on upstream Clear Linux version 13180. A default :file:`builder.conf` - file will be created (if one is not already present in your workspace) - along with a :file:`mix-bundles` directory. + based on the latest released upstream Clear Linux version. A default + :file:`builder.conf` file will be created (if one is not already present in + your workspace), along with several version and tracking files, and two + bundle directories: :file:`local-bundles` and :file:`upstream-bundles`. + (We'll get to how these files and directories are used later.) - If you intend to build a mix with your own custom RPMs, run: + If you wish to start with a different version of upstream Clear Linux, or a + different initial mix version, these can be specified as optional flags. .. code-block:: bash - # sudo mixer init --clear-version 13180 --mix-version 10 --local-rpms + # mixer init --clear-version 21060 --mix-version 100 - This creates :file:`local` and :file:`rpms` directories in your mix, and - adds their paths to the generated :file:`builder.conf`. (For more + + Additionally, if you intend to build a mix with your own custom RPMs, you can + pass the optional :option:`--local-rpms` flag. + + .. code-block:: bash + + # mixer init --local-rpms + + This creates :file:`local-yum` and :file:`local-rpms` directories in your + mix, and adds their paths to the generated :file:`builder.conf`. (For more information on using these directories, or setting them up manually, see Step 4 below.) - You can easily build a mix that includes all Clear bundles with no - modifications using the following command: + If you know you want to include all upstream Clear bundles in your mix, you + can easily add them all to your mix during initialization with the optional + :option:`--all-upstream` flag. .. code-block:: bash - # sudo mixer init --clear-version 13180 --mix-version 10 --all + # mixer init --all-upstream + + Finally, you may find it useful to track the contents of your mixer working + directory with a Git repository. This can be a great way to track changes to + your mix content, or revert to earlier versions should something go wrong. + Mixer can set this up automatically by passing the optional :option:`--git` + flag. + + .. code-block:: bash + + # mixer init --git + + .. note:: + Any or all of the above optional flags can be used at the same time. + For example: + + .. code-block:: bash + + # mixer init --clear-version 21060 --mix-version 100 --local-rpms --all-upstream --git #. **Configure builder.conf**. Edit the :file:`builder.conf` as needed. @@ -64,11 +94,13 @@ Current mixing workflow workspace directory, but the :option:`--config` option exists to specify where the file is if you want to store it elsewhere. - Note there are different sections of :file:`builder.conf`. The - ``[Builder]`` section provides the mixer tools with the required - configuration options, and defines where generated bundles and updated - metadata should be published. The ``[swupd]`` section is used by - swupd-server to create an update with specific update parameters. + Note there are different sections of :file:`builder.conf`. The ``[Mixer]`` + section contains configuration values for how the mixer tool deals with + bundles and keeps track of what is in your mix. The ``[Builder]`` section + provides the mixer tools with the required configuration options for + building your mix, and defines where generated update metadata should be + published. The ``[swupd]`` section is used by swupd-server to create update + content with specific update parameters. Edit the template configuration file according to your needs. For this example, your :file:`builder.conf` should look similar to the example @@ -77,14 +109,15 @@ Current mixing workflow .. code-block:: console - # vim /etc/bundle-chroot-builder/builder.conf: + [Mixer] + LOCAL_BUNDLE_DIR=/home/clr/mix/local-bundles [Builder] SERVER_STATE_DIR=/home/clr/mix/update BUNDLE_DIR=/home/clr/mix/mix-bundles YUM_CONF=/home/clr/mix/.yum-mix.conf CERT=/home/clr/mix/Swupd_Root.pem - VERSIONS_PATH=/home/clr/mix/ + VERSIONS_PATH=/home/clr/mix [swupd] BUNDLE=os-core-update @@ -92,14 +125,25 @@ Current mixing workflow VERSIONURL= FORMAT=1 + [Server] + debuginfo_banned=true + debuginfo_lib=/usr/lib/debug/ + debuginfo_src=/usr/src/debug/ + + The ``LOCAL_BUNDLE_DIR`` is where local bundle definition files are stored. + These include any new, original bundles you create, as well as edited + versions of upstream Clear bundles. (More on this in Step 4 below.) + The ``SERVER_STATE_DIR`` is where the mixed content is output. This is automatically created for you by the mixer. You can set this directory to any location, but we will use the workspace directory for this example. The same applies for ``BUNDLE_DIR``. This directory is - generated for you in the location specified in the :file:`builder.conf`, - in this case ``/home/clr/mix/mix-bundles``. This is where the bundle - definitions are stored for your mix, and it is where the chroot-builder - looks to know what bundles must be installed. + generated for you in the location specified in the :file:`builder.conf`, in + this case ``/home/clr/mix/mix-bundles``. If you are using the legacy + chroot-builder, this directory is where the bundle definition files are + temporarily stored while building chroots. By default, this directory is not + generated until it is needed, and is not generated at all if using the new + chroot-builder built into mixer. (More on this below in Step 8.) The :file:`.yum-mix.conf` file defined in ``YUM_CONF`` is auto-generated, along with the ``CERT`` file, :file:`Swupd_Root.pem`. The yum configuration @@ -125,45 +169,46 @@ Current mixing workflow an easy way to host the content. These URLs are embeded in images created for your mix. They are where ``swupd-client`` will look to figure out if there is a new version available, and the location from which to download - the updated content. Think of these as the equivalent of the `Clear Linux update page`_ used by Clear Linux, but for your derivative mix. + the updated content. Think of these as the equivalent of the + `ClearLinux update page`_ used by Clear Linux, but for your derivative mix. To learn more about the ``FORMAT`` option, refer to the "Format Version" section at the bottom of this document, and `Format Bumps`_ on the Clear Linux wiki. For now, leave the ``FORMAT`` value alone and do not increment it. - The mix version and Clear version will come from two state files: - :file:`.mixversion` and :file:`.clearversion`, both of which will be + The mix version and upstream Clear version will come from two state files: + :file:`mixversion` and :file:`upstreamversion`, both of which will be created for you when you set up the workspace. They will be created in the directory defined by the ``VERSIONS_PATH``. -.. _step-four: - #. **Create/locate RPMs for mix.**. (Steps 4 through 6 are necessary only - if you want to add your own RPMs to the Mix. If you are working only with Clear bundles, then skip to Step 7.) + if you want to add your own RPMs to the Mix. If you are working only with + Clear bundles, then skip to Step 7.) If you are creating RPMs from scratch, you can use ``autospec``, ``mock``, ``rpmbuild``, etc. to build them. If they are not built on Clear, - make sure your configuration and toolchain builds them correctly for Clear, or there is no guarantee they will be compatible. + make sure your configuration and toolchain builds them correctly for Clear, + or there is no guarantee they will be compatible. -#. **Import RPMs into workspace**. Create an :file:`rpms` directory in your - workspace (for example :file:`/home/clr/mix/rpms`), and copy the RPMs you - want into that directory. Next, add the following to your +#. **Import RPMs into workspace**. Create a :file:`local-rpms` directory in your + workspace (for example :file:`/home/clr/mix/local-rpms`), and copy the RPMs + you want into that directory. Next, add the following to your :file:`builder.conf`: .. code-block:: bash - RPMDIR=/home/clr/mix/rpms + LOCAL_RPM_DIR=/home/clr/mix/local-rpms Mixer will look in this directory for RPMs to build a local RPM repo for yum to use. #. **Create a local RPM repo**. Create an empty directory in your workspace - named :file:`local` and add the path in your :file:`builder.conf`: + named :file:`local-yum` and add the path in your :file:`builder.conf`: .. code-block:: bash - REPODIR=/home/clr/mix/local + LOCAL_REPO_DIR=/home/clr/mix/local-yum Once these values are configured, you can generate the yum repo by running the following command: @@ -173,47 +218,155 @@ Current mixing workflow # sudo mixer add-rpms After the tool exits, you should see your RPMs and a repodata directory in - :file:`/home/clr/mix/local`. If the RPMs are not all in this :file:`local` - directory, check to make sure that they are indeed valid RPM files and not - corrupt. + :file:`/home/clr/mix/local-yum`. If the RPMs are not all in this + :file:`local-yum` directory, check to make sure that they are indeed valid + RPM files and not corrupt. -#. **Update/Add bundle definitions**. You can easily add bundles to your mix - by running: +#. **Add/remove/edit/list the bundles in your mix**. The bundles in your mix are + specified in the Mix Bundle List. This list is stored as a flat file called + :file:`mixbundles` in the directory defined by the ``VERSIONS_PATH`` variable + in :file:`builder.conf`. This file is generated automatically during + initialization, and is read from and written to by mixer when you use it to + work with bundles. + + You can view what bundles are already in your mix by running: .. code-block:: bash - # sudo mixer bundle add bundle1,bundle2,... + # mixer bundle list - This command copies the specified bundle defintion files from your - configured upstream version of Clear Linux (:file:`.clearversion`) into - your :file:`mix-bundles` directory. + This will show you a list of every bundle in your mix. Bundles are capable of + including other bundles, and those bundles can themselves include other + bundles. When you list the bundles in your mix this way, mixer will + automatically recurse through these includes and show you every single bundle + that will end up in your mix. - Behind the scenes, mixer uses a local cache of the upstream Clear Linux - bundle definitions. These are stored in the - :file:`.mixer/upstream-bundles/clr-bundles-{VER}/bundles/` directory in - your workspace. Do *not* modify things in this directory; it is simply a - mirror for the tool to use. However, you can refer to the files in this directory to see what bundles are available, or the format these files should have. + If you see a bundle in the list that you weren't expecting, odds are it was + included by something you added in. To get a better view at how a bundle + ended up in your mix, you can pass the :option:`--tree` flag: - To define your bundles: + .. code-block:: bash - #. Navigate to the :file:`mix-bundles/` directory. - #. Make any needed modifications to the bundle set. - #. Commit the result: + # mixer bundle list --tree - .. code-block:: bash + This will print a tree view of your Mix Bundle List, visually showing what + each bundle includes. - $ git add . - $ git commit -s -m 'Update bundles for mix #' + Bundles fall into two categories: **upstream** and **local**. Upstream + bundles are those provided by Clear Linux. Local bundles are bundles you've + created yourself, or edited versions of upstream bundles. - While using Git is optional, with Git history, mixes are easy to revert - or refer to in the future if something goes wrong with a new mix. If - you're just testing this out, or if you really do not want to mess with - Git, you can ignore committing for now. + Upstream bundle definition files are downloaded and cached for you + automatically by mixer, and are stored in the :file:`upstream-bundles` + directory created in your working directory. Do *not* modify things in this + directory; it is simply a mirror for the tool to use. The tool automatically + caches the bundles for your configured version of Clear Linux (in your + :file:`upstreamversion` file), and cleans up old versions once they are no + longer needed. You can see what upstream bundles are available by running: - To add your own bundle, create a bundle definition file in the correct - format in the :file:`mix-bundles` directory (you can refer to an existing - bundle, like :file:`mix-bundles/os-core-update`, for formatting). Be sure that the bundle name you choose does not conflict with another bundle. - Add your package name(s) in the bundle definition file to tell it what package(s) must be installed as part of that bundle. + .. code-block:: bash + + # mixer bundle list upstream + + Local bundle definition files live in the :file:`local-bundles` directory. + The location of this directory is specified by ``LOCAL_BUNDLE_DIR`` in your + :file:`builder.conf`. For this example, this is + :file:`/home/clr/mix/local-bundles`. You can see what local bundles are + available by running: + + .. code-block:: bash + + # mixer bundle list local + + With either of the above commands, you can pass the :option:`--tree` flag to + see a tree view of what other bundles each bundle includes. + + When looking for a bundle definition file, **mixer always checks local + bundles first, then upstream**. As such, bundles found in + :file:`local-bundles` will always take precedence to upstream bundles of the + same name. This is how "editing" an upstream bundle works; the local, edited + version overrides the version found upstream. (More on editing bundles in + a moment.) + + You can easily **add bundles** to your mix by running: + + .. code-block:: bash + + # mixer bundle add bundle1 [bundle2...] + + This command adds the bundles you specify to your Mix Bundle List + (:file:`mixbundles`). For each bundle you add, mixer checks your local and + upstream bundles to make sure that the bundle you're adding actually exists. + If any cannot be found, an error will be reported. When mixer adds a bundle, + it will tell you whether it was found in local or upstream. You can also see + this information when you run :command:`mixer bundle list`. + + To **remove a bundle** from your mix, run: + + .. code-block:: bash + + # mixer bundle remove bundle1 [bundle2...] + + This command will remove the bundles you specify from your Mix Bundle List + (:file:`mixbundles`). By default, it does not remove the bundle definition + file from your local bundles. If you would like to completely remove a + bundle, including its local bundle definition file, the :option:`--local` + flag can be passed. By default, removing a local bundle file this way will + remove it from your mix as well. If you wish to *only* remove the local + bundle definition file, you can also pass the :option:`--mix=false` flag. + Please note that if you remove a local bundle that was an edited version of + upstream, and that bundle is still in your mix, your mix will now be + referencing the original upstream version of the bundle. If you remove a + bundle that was *only* found locally and still leave the bundle in your Mix + Bundles List, there will no longer be any valid bundle definition file to + refer to, and mixer will produce an error. + + To **edit a bundle definition file**, run: + + .. code-block:: bash + + # mixer bundle edit bundle1 [bundle2...] + + If the bundle is found in your local bundles, mixer will edit this bundle + definition file. If instead the bundle is only found upstream, mixer will + copy the bundle definition file from upstream into your :file:`local-bundles` + directory first. In either case, mixer will launch your default editor to + edit the file. When the editor closes, mixer automatically validates the + edited bundle file, and reports any errors it encounters. If it does find an + error, you have the option of continuing to edit the file as-is, revert and + edit, or skip and keep going to the next bundle. If you skip a file, a backup + of the original file is saved with the ``.orig`` suffix. Because mixer always + checks your local bundles first, edited copies of an upstream bundle will + always take precedence over their upstream counterpart. + + This same command is used to create a totally **new bundle**: if the bundle + name you specify is not found upstream, a blank template is generated in + :file:`local-bundles` with the correct filename. The editor is again launched + for you to fill out the bundle, and validation is performed on exiting. Add + your package name(s) in the bundle definition file to tell it what package(s) + must be installed as part of that bundle. + + Mixer will do basic **validation** on all bundles when they are used + throughout the system: it will check that the bundle syntax is valid and can + be parsed, and that the bundle file has a valid name. If you would like to + manually run this validation on a bundle, you can run: + + .. code-block:: bash + + # mixer bundle validate bundle1 [bundle2...] + + This command has an optional :option:`--strict` flag, which additionally + checks that the rest of the bundle header fields can be parsed and are + non-empty, and that the bundle header ``Title`` field and the bundle filename + match. + + .. note:: + If you initialized your workspace to be tracked as a Git repository + (:command:`mixer init --git`), you may find it useful to apply a git + commit after modifying what bundles are in your Mix Bundle List or + editing a bundle definition file. All of the above :command:`mixer + bundle` commands support an optional :option:`--git` flag that will + automatically apply a git commit when they are finished. #. **Build the bundle chroots**. To build all of the ``chroots`` that are based on the bundles you defined, run the following command in your @@ -223,9 +376,22 @@ Current mixing workflow # sudo mixer build chroots - If you have many bundles defined for your mix, this step might take some + If you have many bundles in your mix, this step might take some time. + By default, mixer will use the legacy chroot-builder. In this mode, the + bundle definition files for the bundles in your mix will be automatically + gathered into a :file:`mix-bundles` directory in the location specified by + ``BUNDLE_DIR`` in your :file:`builder.conf`. **Do not edit these files**. + Mixer will automatically clear out any contents of this directory before + populating it on-the-fly as chroots are built. + + Mixer now has a new chroot-builder built into the mixer tool itself. While + this is currently an experimental feature, you can (and should) use the new + chroot-builder by passing the :option:`--new-chroots` flag. The legacy + chroot-builder will soon be deprecated, and mixer will use the new version + automatically. + #. **Create update**. In the workspace, run: .. code-block:: bash @@ -237,6 +403,13 @@ Current mixing workflow located in :file:`/home/clr/mix/update/www/{}`, where ```` is the mix version you defined (10 by default). + By default, mixer will use the legacy swupd-server to generate the update + content. Mixer now has a new implementation built into the mixer tool itself. + While this is currently an experimental feature, you can (and should) use the + new swupd-server by passing the :option:`--new-swupd` flag. The legacy + swupd-server will soon be deprecated, and mixer will use the new version + automatically. + All content to make a fully usable mix will be created by this step, but note that only *zero packs* are automatically generated. Zero packs are the content needed to go from nothing to the mix version you just built @@ -252,7 +425,7 @@ Current mixing workflow from ``PAST_VERSION`` to ``MIX_VERSION``. If your ``STATE_DIR`` is in a different location, be sure to specify the location with the ``-S`` option. For the first build, no delta packs can be created because the - "update" is from version 0. Version 0 impicitly has no content, thus no + "update" is from version 0. Version 0 implicitly has no content, thus no deltas can be generated. For subsequent builds, :file:`mixer-pack-maker.sh` can be run to generate delta content between them (for example: 10 to 20). @@ -269,10 +442,10 @@ Current mixing workflow For reference, you can inspect the ``ister`` config file that `Clear Linux uses`_ for its releases. - Note that mixer automatically looks for a file named :file:`release-image- - config.json`, but you can choose whatever name you want. To use a - different name, simply pass the :option:`--template path/to/file.config` - flag when creating your image. + Note that mixer automatically looks for a file named + :file:`release-image-config.json`, but you can choose whatever name you want. + To use a different name, simply pass the + :option:`--template path/to/file.config` flag when creating your image. Edit the config file to include all bundles that you want *preinstalled* into your image. The rest of the bundles in your mix will be available to your users via: @@ -316,17 +489,65 @@ Current mixing workflow Creating your next mix version ============================== -**Update the next Mix version info**. Update the :file:`.mixversion` file to -the next version number you want to build. +**Update the next Mix version info**. To increment your mix version number for +your next mix, run: -From this point you can iterate through the instructions , starting again at -:ref:`step 4 ` and making modifications as needed. For example: + .. code-block:: bash + + # mixer versions update + +This command will automatically update your mix version (stored in the +:file:`mixversion` file), incrementing it by 10. If you'd like to increment by a +different amount, the :option:`--increment` flag can be used: + + .. code-block:: bash + + # mixer versions update --increment 100 + +Alternatively, if you want set your mix version to a specific value, you can +do so with the :option:`--mix-version` flag: + + .. code-block:: bash + + # mixer versions update --mix-version 200 + +Please note that the :command:`mixer versions update` command does not allow you +to set your mix version to something lower than its current value. This is +because your mix version is expected to always increase, even if the new mix is +undoing an earlier change. (If you've been tracking your working directory with +Git, it is possible to restore your mix to an earlier state, but be careful of +"rewriting history" if you are already publishing your mix content to users.) + +From this point you can iterate through the instructions, starting again at +Step 4 and making modifications as needed. For example: - Add/remove/modify bundles - ``sudo mixer build chroots`` - ``sudo mixer build update`` - (Optionally) ``sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update`` +If you want to update the upstream version of Clear Linux on which your mix is +based, you can do so using the :option:`--upstream-version` flag: + + .. code-block:: bash + + # mixer versions update --upstream-version 21070 + +This command also accepts the keyword "latest": :option:`--upstream-version +latest`. This will set your upstream version to the latest released version of +upstream Clear Linux within the same format version. The :command:`mixer +versions update` command does not allow you to set your upstream version to a +value that would cross an upstream format boundary, as this would require a +"format bump" build, which is currently a manual process. You can read more +about format versions below. + +If you simply wish to learn which mix version or upstream version you currently +are on, you can run: + + .. code-block:: bash + + # mixer versions + Format Version ************** @@ -355,4 +576,4 @@ require a format increment. .. _Clear Linux update page: https://cdn.download.clearlinux.org/update/ .. _Format Bumps: https://github.com/clearlinux/swupd-server/wiki/Format-Bumps -.. _Clear Linux uses: https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json \ No newline at end of file +.. _Clear Linux uses: https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json From 73f21faff863ee6dc8d88e3ea507dfe03939991e Mon Sep 17 00:00:00 2001 From: Rodrigo Caballero Date: Tue, 6 Mar 2018 11:21:14 -0600 Subject: [PATCH 4/8] Rewrite guide to improve clarity, flow, markup, and consistency. The rewrite includes: Explicit commands for important options. Use of lists to improve the clarity of the configuration variables content. Removal of command prompts to enable DBT. Consistent use of terminology. Use of concrete examples to improve the clarity of the commands. Improved content flow and navigation with linked workflow overviews. Added cross-references. Removed ambiguous relative pronouns and punctuation. Removed passive voice whenever possible. Changed verbs to present tense for consistency and style guide compliance. Optimized headings for search engines. Signed-off-by: Rodrigo Caballero --- .../clear-linux/guides/maintenance/mixer.rst | 1184 ++++++++++------- 1 file changed, 686 insertions(+), 498 deletions(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index d03f3b4b..341f65c0 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -3,577 +3,765 @@ Use mixer tool ############## -*Mixing* refers to composing an operating system for specific use cases. -While the default Clear Linux* OS for Intel® Architecture provides options -to install bundles for various server capabilities, some developers may wish -to either augment the operating system itself with functionality from their -own packages, or modify the structure of current bundles to cater to their -particular needs. +*Mixing* refers to composing an operating system for specific use cases. While +the default |CLOSIA| provides options to install bundles for various server +capabilities, some developers may wish to either augment the operating system +itself with functionality from their own packages or modify the structure of +current bundles to cater to their particular needs. Prerequisites ************* -To start working with the mixer tools, you'll need a recent image of Clear -Linux OS for Intel Architecture with the following bundle installed. If you -don't have it already, you can add it with the :command:`swupd bundle-add` -command like so: +To start working with the mixer tools, you need a recent image of |CL| with +the `mixer` bundle installed. If the bundle is not yet installed, you can add it +with the :command:`swupd bundle-add` command as follows: .. code-block:: bash - # swupd bundle-add mixer + sudo swupd bundle-add mixer Current mixing workflow *********************** -#. **Create a workspace**. Create an empty directory in your Clear image to - use as a "workspace" for mixing. For these steps, we assume your workspace - location is :file:`/home/clr/mix`. +There are two different workflows to create your own mix. -#. **Generate the starting point for your mix**. In your workspace, run: +The following workflow applies if the mix includes your own +:abbr:`RPMs (RPM Package Manager files)`: + +#. `Create a workspace`_ +#. `Generate the starting point for your mix`_ +#. `Edit builder.conf`_ +#. `Create or locate RPMs for the mix`_ +#. `Import RPMs into workspace`_ +#. `Create a local RPM repo`_ +#. `List, edit, create, add, remove, or validate bundles`_ +#. `Build the bundle chroots`_ +#. `Create an update`_ +#. `Create an image`_ + +Alternatively, the following workflow applies if the mix only uses |CL| +bundles. + +#. `Create a workspace`_ +#. `Generate the starting point for your mix`_ +#. `Edit builder.conf`_ +#. `List, edit, create, add, remove, or validate bundles`_ +#. `Build the bundle chroots`_ +#. `Create an update`_ +#. `Create an image`_ + +The following sections contain detailed information on every step of +these workflows. + +Create a workspace +****************** + +Create an empty directory in your |CL| image to use as a **workspace** for +mixing with the following command: + +.. code-block:: bash + + mkdir /home/clr/mix + +This guide assumes your workspace location is :file:`/home/clr/mix`. + +Generate the starting point for your mix +**************************************** + +In your workspace, initialize mixer with the following command: + +.. code-block:: bash + + sudo mixer init + +This command initializes your workspace for you to make a mix at version 10 +based on the latest released upstream |CL| version. If a :file:`builder.conf` +file is not already present in your workspace, mixer creates a default configuration +file along with several version and tracking files and two bundle directories: +:file:`local-bundles` and :file:`upstream-bundles`. + +If you wish to start with a different version of upstream |CL| or a +different initial mix version, these options can be specified as flags, for example: + +.. code-block:: bash + + sudo mixer init --clear-version 21060 --mix-version 100 + + +Additionally, to build a mix with your own custom RPMs, use the optional + :option:`--local-rpms` flag, for example: + +.. code-block:: bash + + sudo mixer init --local-rpms + +This command creates the :file:`local-yum` and :file:`local-rpms` directories in your +mix workspace and adds their paths to the generated :file:`builder.conf`. For more +information on using these directories or setting them up manually, see +`Create or locate RPMs for the mix`_. + +If all upstream |CL| bundles are part of the mix, you can easily add them all +during initialization with the optional :option:`--all-upstream` flag, for +example: + +.. code-block:: bash + + sudo mixer init --all-upstream + +Finally, you may want to track the contents of your mixer workspace with a Git +repository. This is a great way to track changes to your mix's content or to +revert to earlier versions if something goes wrong. Mixer can set this up +automatically with the optional :option:`--git` flag, for example: + +.. code-block:: bash + + sudo mixer init --git + +.. note:: + You can use any or all of the above optional flags at the same time, for example: .. code-block:: bash - # mixer init + sudo mixer init --clear-version 21060 --mix-version 100 --local-rpms --all-upstream --git - This initializes your workspace so that you can make a mix at version 10 - based on the latest released upstream Clear Linux version. A default - :file:`builder.conf` file will be created (if one is not already present in - your workspace), along with several version and tracking files, and two - bundle directories: :file:`local-bundles` and :file:`upstream-bundles`. - (We'll get to how these files and directories are used later.) +Edit builder.conf +***************** - If you wish to start with a different version of upstream Clear Linux, or a - different initial mix version, these can be specified as optional flags. +To configure the mixer tool, you must edit the :file:`builder.conf` as needed. - .. code-block:: bash +The file :file:`builder.conf` is read automatically from the current workspace +directory. Use the :option:`--config` flag during initialization to specify a +alternate path to the file as needed. - # mixer init --clear-version 21060 --mix-version 100 +The :file:`builder.conf` file has different sections, for example: +* The `[Builder]` section provides the mixer tools with the required + configuration options. This section defines the path where the generated + bundles and update metadata are published. - Additionally, if you intend to build a mix with your own custom RPMs, you can - pass the optional :option:`--local-rpms` flag. +* The `[swupd]` section contains specific update parameters. The + :abbr:`swupd-server (software update server)` creates an update using + said specific update parameters. - .. code-block:: bash +Edit the configuration file according to your needs with the command: - # mixer init --local-rpms +.. code-block:: bash - This creates :file:`local-yum` and :file:`local-rpms` directories in your - mix, and adds their paths to the generated :file:`builder.conf`. (For more - information on using these directories, or setting them up manually, see - Step 4 below.) + sudo vim /etc/bundle-chroot-builder/builder.conf - If you know you want to include all upstream Clear bundles in your mix, you - can easily add them all to your mix during initialization with the optional - :option:`--all-upstream` flag. +Your version of the :file:`builder.conf` file should resemble the +following example: - .. code-block:: bash +.. code-block:: console - # mixer init --all-upstream + [Mixer] + LOCAL_BUNDLE_DIR=/home/clr/mix/local-bundles - Finally, you may find it useful to track the contents of your mixer working - directory with a Git repository. This can be a great way to track changes to - your mix content, or revert to earlier versions should something go wrong. - Mixer can set this up automatically by passing the optional :option:`--git` - flag. + [Builder] + SERVER_STATE_DIR=/home/clr/mix/update + BUNDLE_DIR=/home/clr/mix/mix-bundles + YUM_CONF=/home/clr/mix/.yum-mix.conf + CERT=/home/clr/mix/Swupd_Root.pem + VERSIONS_PATH=/home/clr/mix - .. code-block:: bash + [swupd] + BUNDLE=os-core-update + CONTENTURL= + VERSIONURL= + FORMAT=1 - # mixer init --git + [Server] + debuginfo_banned=true + debuginfo_lib=/usr/lib/debug/ + debuginfo_src=/usr/src/debug/ - .. note:: - Any or all of the above optional flags can be used at the same time. - For example: +The following variables require further explanation: - .. code-block:: bash +* The `LOCAL_BUNDLE_DIR` variable sets the path where mixer stores the local + bundle definition files. These bundle definition files include any new, + original bundles you create and edited versions of upstream |CL| bundles. - # mixer init --clear-version 21060 --mix-version 100 --local-rpms --all-upstream --git +* The `SERVER_STATE_DIR` variable sets the path for the output of the mix + content. Mixer automatically creates the path for you but the path can be + set to any location. In this example, we use the workspace directory. -#. **Configure builder.conf**. Edit the :file:`builder.conf` as needed. +* The `BUNDLE_DIR` sets the path where mixer temporarily stores the bundle + definition files while building chroots. Only the legacy chroot-builder uses + this path. By default, mixer does not generate this directory until needed. + In our example, the path is set to :file:`/home/clr/mix/mix-bundles`. The + new chroot-builder does not generate the folder at all. - The :file:`builder.conf` file will be read automatically from the current - workspace directory, but the :option:`--config` option exists to specify - where the file is if you want to store it elsewhere. +* The `YUM_CONF` variable sets path where mixer automatically generates the + :file:`.yum-mix.conf` yum configuration file. The yum configuration file + points the chroot-builder to the path where the RPMs are stored. - Note there are different sections of :file:`builder.conf`. The ``[Mixer]`` - section contains configuration values for how the mixer tool deals with - bundles and keeps track of what is in your mix. The ``[Builder]`` section - provides the mixer tools with the required configuration options for - building your mix, and defines where generated update metadata should be - published. The ``[swupd]`` section is used by swupd-server to create update - content with specific update parameters. +* The `CERT` variable sets the path where mixer stores the + :file:`Swupd_Root.pem` certificate file. The chroot-builder needs the + certificate file to sign the root :file:`Manifest.MoM` file to provide + security for content verification. The value of the `CERT` variable can + point to a different certificate. The chroot-builder inserts the + certificate specified here in the + :file:`/os-core-update/usr/share/clear/update-ca/` path. The software + update client uses this certificate to + verify the :file:`Manifest.MoM` file's signature. For now, we **strongly** + recommend not to modify this line as the certificate `swupd` expects needs + a very specific configuration to sign and verify properly. Mixer + automatically generates the certificate and signs the + :file:`Manifest.MoM` file providing security for your created update + content. - Edit the template configuration file according to your needs. For this - example, your :file:`builder.conf` should look similar to the example - below, with both URL variables set to the domain or IP of the update - server: +* The `CONTENTURL` and `VERSIONURL` variables set the domain or IP address + where mixer serves your updated content and the corresponding version. You + can set the `CONTENTURL` and `VERSIONURL` URL variables to the domain or IP- + address of the update server. The `SERVER_STATE_DIR` directory is hosted + here: :file:`/home/clr/mix/update/www`. Creating a symlink to the directory + in your server webdir is an easy way to host the content. These URLs are + embedded in images created for your mix. The `swupd-client` looks at these + URLs to figure out if there is a new version available and the location from + which to download the updated content. Think of these links as the + equivalent to the |CL| `update page`_ but for your mix. + +* The `FORMAT` variable relates to the format bump. To learn more about the + `FORMAT` option, see :ref:`mixer-format` and the `format bumps wiki`_. For + now, leave the `FORMAT` value unchanged. + +* The `VERSIONS_PATH` variable set the path for the mix version and upstream + |CL| version two state files: :file:`mixversion` and + :file:`upstreamversion`. Mixer creates both files for you when you set up + the workspace. + +.. note:: If you are working only with |CL| bundles, then + skip to `List, edit, create, add, remove, or validate bundles`_. + +Create or locate RPMs for the mix +********************************* + +If you create RPMs from scratch, you can use `autospec`, `mock`, `rpmbuild`, +or similar tools to build them. If the RPMs are not built on |CL|, ensure your +configuration and tool-chain builds them correctly for Clear or there is no +guarantee they will be compatible. For more information on building the RPMs +properly visit our `build RPMs instructions`_. + +Import RPMs into workspace +************************** + +#. Create a :file:`local-rpms` directory in your workspace, for example + :file:`/home/clr/mix/local-rpms`. + +#. Copy the RPMs into the created directory. + +#. Add the following line to your :file:`builder.conf` file: .. code-block:: console - [Mixer] - LOCAL_BUNDLE_DIR=/home/clr/mix/local-bundles - - [Builder] - SERVER_STATE_DIR=/home/clr/mix/update - BUNDLE_DIR=/home/clr/mix/mix-bundles - YUM_CONF=/home/clr/mix/.yum-mix.conf - CERT=/home/clr/mix/Swupd_Root.pem - VERSIONS_PATH=/home/clr/mix - - [swupd] - BUNDLE=os-core-update - CONTENTURL= - VERSIONURL= - FORMAT=1 - - [Server] - debuginfo_banned=true - debuginfo_lib=/usr/lib/debug/ - debuginfo_src=/usr/src/debug/ - - The ``LOCAL_BUNDLE_DIR`` is where local bundle definition files are stored. - These include any new, original bundles you create, as well as edited - versions of upstream Clear bundles. (More on this in Step 4 below.) - - The ``SERVER_STATE_DIR`` is where the mixed content is output. This - is automatically created for you by the mixer. You can set this - directory to any location, but we will use the workspace directory for - this example. The same applies for ``BUNDLE_DIR``. This directory is - generated for you in the location specified in the :file:`builder.conf`, in - this case ``/home/clr/mix/mix-bundles``. If you are using the legacy - chroot-builder, this directory is where the bundle definition files are - temporarily stored while building chroots. By default, this directory is not - generated until it is needed, and is not generated at all if using the new - chroot-builder built into mixer. (More on this below in Step 8.) - - The :file:`.yum-mix.conf` file defined in ``YUM_CONF`` is auto-generated, - along with the ``CERT`` file, :file:`Swupd_Root.pem`. The yum configuration - file is needed for the chroot-builder to know where the RPMs are hosted, - and the certificate file is needed to sign the root Manifest to provide - security for content verification. - - You can change the ``CERT=/path/to/cert`` line to point to a different - certificate. The chroot builder inserts the certificate specified here - in ``/os-core-update/usr/share/clear/update-ca/``. This is the certificate - used by the software update client to verify the :file:`Manifest.MoM` - signature. For now, we *highly* recommend that you do not modify this - line, as the certificate that swupd expects needs a very specific - configuration to sign and verify properly. The certificate is - automatically generated, and the :file:`Manifest.MoM` is signed - automatically as well, providing security for the updated content that - you create. - - The ``CONTENTURL`` and ``VERSIONURL`` should be set to the domain or IP - address where your updated content will be served. This is the location - that hosts the :file:`/home/clr/mix/update/www` (``SERVER_STATE_DIR``) - directory. Creating a symlink to the directory in your server webdir is - an easy way to host the content. These URLs are embeded in images created - for your mix. They are where ``swupd-client`` will look to figure out if - there is a new version available, and the location from which to download - the updated content. Think of these as the equivalent of the - `ClearLinux update page`_ used by Clear Linux, but for your derivative mix. - - To learn more about the ``FORMAT`` option, refer to the "Format Version" - section at the bottom of this document, and `Format Bumps`_ on the Clear - Linux wiki. For now, leave the ``FORMAT`` value alone and do not increment - it. - - The mix version and upstream Clear version will come from two state files: - :file:`mixversion` and :file:`upstreamversion`, both of which will be - created for you when you set up the workspace. They will be created in the - directory defined by the ``VERSIONS_PATH``. - -#. **Create/locate RPMs for mix.**. (Steps 4 through 6 are necessary only - if you want to add your own RPMs to the Mix. If you are working only with - Clear bundles, then skip to Step 7.) - - If you are creating RPMs from scratch, you can use ``autospec``, ``mock``, - ``rpmbuild``, etc. to build them. If they are not built on Clear, - make sure your configuration and toolchain builds them correctly for Clear, - or there is no guarantee they will be compatible. - -#. **Import RPMs into workspace**. Create a :file:`local-rpms` directory in your - workspace (for example :file:`/home/clr/mix/local-rpms`), and copy the RPMs - you want into that directory. Next, add the following to your - :file:`builder.conf`: - - .. code-block:: bash - LOCAL_RPM_DIR=/home/clr/mix/local-rpms - Mixer will look in this directory for RPMs to build a local RPM repo for - yum to use. +Mixer uses this directory to find the RPMs to build a local RPM repo for +yum to use. -#. **Create a local RPM repo**. Create an empty directory in your workspace - named :file:`local-yum` and add the path in your :file:`builder.conf`: +Create a local RPM repo +*********************** - .. code-block:: bash +#. Create an empty directory in your workspace named :file:`local-yum`. +#. Add the path to your :file:`builder.conf` file: + + .. code-block:: console LOCAL_REPO_DIR=/home/clr/mix/local-yum - Once these values are configured, you can generate the yum repo by - running the following command: +#. With these values configured, generate the yum repo with the following command: .. code-block:: bash - # sudo mixer add-rpms + sudo mixer add-rpms - After the tool exits, you should see your RPMs and a repodata directory in - :file:`/home/clr/mix/local-yum`. If the RPMs are not all in this - :file:`local-yum` directory, check to make sure that they are indeed valid - RPM files and not corrupt. +After the tool exits, you should see the RPMs and a repository data directory in +:file:`/home/clr/mix/local-yum`. If the RPMs are not all in this +:file:`local-yum` directory, check to ensure they are valid RPM files and not +corrupt. -#. **Add/remove/edit/list the bundles in your mix**. The bundles in your mix are - specified in the Mix Bundle List. This list is stored as a flat file called - :file:`mixbundles` in the directory defined by the ``VERSIONS_PATH`` variable - in :file:`builder.conf`. This file is generated automatically during - initialization, and is read from and written to by mixer when you use it to - work with bundles. +List, edit, create, add, remove, or validate bundles +**************************************************** - You can view what bundles are already in your mix by running: +The bundles in the mix are specified in the mix bundle list. Mixer stores +this list as a flat file called :file:`mixbundles` in the path set in the +`VERSIONS_PATH` variable of the :file:`builder.conf` file. Mixer generates +the mix bundle list file automatically during initialization. Mixer reads +and writes the bundle list file when you change the bundles of the mix. + +List the bundles in the mix +=========================== + +To view the bundles already in the mix, enter the following command: + +.. code-block:: bash + + sudo mixer bundle list + +The command shows a list of every bundle in the mix. Bundles can include other +bundles and those bundles can themselves include other bundles. When listing +the bundles with this command, mixer automatically recurses through the +includes to show every single bundle in the mix. + +If you see an unexpected bundle in the list, the bundle probably is included +in another bundle. Use the :option:`--tree` flag to get a better view of how a +bundle ended up in the mix, for example: + +.. code-block:: bash + + sudo mixer bundle list --tree + +This command prints a tree view of the mix bundle list explicitly showing each +included bundle. + +Bundles fall into two categories: **upstream** and **local**. + +Upstream bundles are those provided by |CL|. + +Mixer automatically downloads and caches upstream bundle definition files. The +definition files are stored in the :file:`upstream-bundles` directory in the +workspace. Do **not** modify the files in this directory. The directory is +simply a mirror for mixer to use. + +The mixer tool automatically caches the bundles for the |CL| version configured in the +:file:`upstreamversion` file. Mixer also cleans up old versions once they are no +longer needed. See the available upstream bundles with the following command: + +.. code-block:: bash + + sudo mixer bundle list upstream + +Local bundles are bundles you create or edited versions of upstream bundles. + +Local bundle definition files live in the :file:`local-bundles` directory. +The `LOCAL_BUNDLE_DIR` variable sets the path of this directory in your +:file:`builder.conf` configuration file. For this example, the path is +:file:`/home/clr/mix/local-bundles`. See the available local bundles with the +following command: + +.. code-block:: bash + + sudo mixer bundle list local + +Both the local or upstream :command:`bundle list` commands accept the +:option:`--tree` flag to print a tree view explicitly showing each included +bundle. + +Edit the bundles in the mix +=========================== + +**Mixer always checks local bundles first and the upstream bundles second.** + +Therefore, bundles in the :file:`local-bundles` directory always take +precedence over the upstream bundles of the same name. + +This precedence enables the editing of upstream bundles. The local, edited +version of the bundle overrides the bundle version found upstream. + +For example, to edit the `bundle1` definition file, we use the following +command: + +.. code-block:: bash + + sudo mixer bundle edit bundle1 + +If `bundle1` is found in your local bundles, mixer edits the bundle definition +file. If instead `bundle1` is only found upstream, mixer copies the bundle +definition file from upstream into your :file:`local-bundles` directory first. + +In both cases, mixer launches your default editor to edit the file. When the +editor closes, mixer automatically validates the edited bundle file and +reports any errors found. If mixer finds an error, you can edit the file as- +is, revert and edit, or skip and move on to the next bundle. If you skip a +file, mixer saves a backup of the original file with the ``.orig`` suffix. +Because mixer always checks your local bundles first, edited copies of an +upstream bundle always take precedence over their upstream counterpart. You +can edit multiple bundles with the following command: + +.. code-block:: bash + + sudo mixer bundle edit bundle1 bundle2 [bundle3 ...] + +Create bundles for the mix +========================== + +To create a totally **new bundle**, the bundle name you specify cannot exist +upstream. If that is the case, create a `new-bundle` with the following +command: + +.. code-block:: bash + + sudo mixer bundle edit new-bundle + +This command generates a blank template in :file:`local-bundles` with the +:file:`new-bundle` filename. Mixer launches the editor for you to fill out the +bundle and performs validation on exiting. Add your package or packages in the +bundle definition file to define the packages to install as part of the bundle. + +.. note:: + + The :command:`mixer bundle edit` accepts multiple bundles at once. Thus, + you can create multiple new bundles in a single command, for example: .. code-block:: bash - # mixer bundle list + sudo mixer bundle edit new-bundle1 new-bundle2 [new-bundle3 ...] - This will show you a list of every bundle in your mix. Bundles are capable of - including other bundles, and those bundles can themselves include other - bundles. When you list the bundles in your mix this way, mixer will - automatically recurse through these includes and show you every single bundle - that will end up in your mix. +Add bundles to the mix +====================== - If you see a bundle in the list that you weren't expecting, odds are it was - included by something you added in. To get a better view at how a bundle - ended up in your mix, you can pass the :option:`--tree` flag: +Add `bundle1` to your mix easily with the following command: + +.. code-block:: bash + + sudo mixer bundle add bundle1 + +This command adds the bundles you specify to your mix bundles list stored in +the :file:`mixbundles` file. For each bundle you add, mixer checks your local +and upstream bundles to ensure the added bundle actually exists. If mixer +cannot find the bundle, it reports back an error. Additionally, when mixer +adds a bundle, it tells you whether the bundle is local or upstream. +Alternatively, you can learn this information with the +:command:`mixer bundle list` command, see `List the bundles in the mix`_. + +To add multiple bundles at once, use the following command: + +.. code-block:: bash + + sudo mixer bundle add bundle1 bundle2 [bundle3 ...] + +Remove bundles from the mix +=========================== + +Remove `bundle1` from your mix with the following command: + +.. code-block:: bash + + sudo mixer bundle remove bundle1 + +This command remove the `bundle1` from the mix bundle list stored in the +:file:`mixbundles` file. By default, the command does not remove the bundle +definition file from your local bundles. To completely remove a bundle, +including its local bundle definition file, use the following command with the +:option:`--local` flag: + +.. code-block:: bash + + sudo mixer bundle remove --local bundle1 + +By default, removing a local bundle file with this command removes the bundle +from the mix as well. To only remove the local bundle definition file, use the +following command with the :option:`--mix=false` flag: + +.. code-block:: bash + + sudo mixer bundle remove --local --mix=false bundle1 + +If you remove a local edited version of an upstream bundle and keep the bundle +in the mix, the mix then references the original upstream version of the +bundle. + +On the other hand, if you remove a bundle only found locally but keep the +bundle in the mix bundles list, mixer will no longer find a valid bundle +definition file and will produce an error. + +Validate the bundles in the mix +=============================== + +Mixer performs basic **validation** on all bundles when used +throughout the system. + +Mixer checks the validity of the bundle's syntax and name. Mixer also ensures +the bundle can be parsed. Run this validation manually on bundle1 with the +following command: + +.. code-block:: bash + + sudo mixer bundle validate bundle1 + +With the optional :option:`--strict` flag, the command additionally +checks if the rest of the bundle header fields can be parsed, if they are +non-empty, and if the bundle header ``Title`` field and the bundle filename +match. Perform a strict validation of `bundle1` with the following command: + +.. code-block:: bash + + sudo mixer bundle validate --strict bundle1 + +Validate multiple bundles with the following command: + +.. code-block:: bash + + sudo mixer bundle validate bundle1 bundle2 [bundle3 ...] + +Managing bundles with Git +========================= + +If you initialized your workspace to be tracked as a Git repository +with the :command:`mixer init --git` command, it might be useful to apply a git +commit after modifying the mix bundle list or editing a bundle definition file. + +All the :command:`mixer bundle` commands on the previous sections support an +optional :option:`--git` flag. The flag automatically applies a git commit +when the command completes, for example: + +.. code-block:: bash + + sudo mixer bundle remove --git bundle1 + +Build the bundle chroots +************************ + +To build all the ``chroots`` based on the defined bundles, use the following +command in your workspace: + +.. code-block:: bash + + sudo mixer build chroots + +If the mix has many bundles, this step might take some time. + +By default, mixer uses the legacy chroot-builder. In this mode, mixer +automatically gathers the bundle definition files for the bundles in the mix +into a :file:`mix-bundles` directory. The directory's path is set in the +`BUNDLE_DIR` variable in the :file:`builder.conf`. **Do not edit these +files.** Mixer automatically clears out any contents in this directory before +populating it on-the-fly as mixer builds the chroots. + +We have build a new chroot-builder into the mixer tool itself. While this is +currently an experimental feature, you should use the new chroot-builder. To use +the new chroot-builder, use the following command with the +:option:`--new-chroots` flag: + +.. code-block:: bash + + sudo mixer build chroots --new-chroots + +We will soon deprecate the legacy chroot-builder and mixer will use the new version +automatically. + +Create an update +**************** + +Create an update with the following command: + +.. code-block:: bash + + sudo mixer build update + +When the build completes, you can find the mix update content under +:file:`/home/clr/mix/update/www/VER`. In our example, the update content is +found in :file:`/home/clr/mix/update/www/{}`. `` +is the mix version defined, 10 by default. + +By default, mixer uses the legacy `swupd-server` to generate the update +content. We have built a new implementation into the mixer tool itself. While +this is currently an experimental feature, you should use the new swupd- +server. To use the the new swupd-server, use the following command with the +:option:`--new-swupd` flag: + +.. code-block:: bash + + sudo mixer build update --new-swupd + +We will soon deprecate the legacy swupd-server and mixer will use the new version +automatically. + +Mixer creates all the content needed to make a fully usable mix with this +step. However, only *zero packs* are automatically generated. Zero packs are +the content needed to go from nothing to the mix version for which you just +built the content. + +Create optional *delta packs*, which allow the transition from one mix version +to another, with the following command: + +.. code-block:: bash + + sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update + +The pack-maker generates all delta packs for the bundles changed from +`PAST_VERSION` to `MIX_VERSION`. If your `STATE_DIR` is in a different +location, specify the location with the :option:`-S` flag. Mixer cannot create +delta packs for the first build because the update is from version 0. Version +0 implicitly has no content, thus, mixer can generate no deltas. + +For subsequent builds, you can run :file:`mixer-pack-maker.sh` to generate +delta content between them, for example: 10 to 20. + +Create an image +***************** + +Since mixer uses the `ister` tool to create a bootable image from your updated +content, we must first configure the `ister` tool. To configure the image +`ister` creates, we need the `ister` configuration file. Obtain a copy with +the default values from the `ister` package with the following command: + +.. code-block:: bash + + sudo cp /usr/share/defaults/ister/ister.json relase-image-config.json + +For reference, you can inspect the `Clear Linux ister configuration file`_ +used for releases. + +Edit the configuration file to include all bundles you want *preinstalled* in +the image. Users can install the bundles in the mix not included in the +configuration file with the following command: + +.. code-block:: bash + + sudo swupd bundle add + +Keeping the list of bundles in the configuration file small allows for a +smaller image size. For the minimal base image, the list is: + +.. code-block:: console + + "Bundles": ["os-core", "os-core-update", "kernel-native"] + +Next, set the `Version` field to the mix version content mixer should use to +build image. `ister` allows you to build an image from any mix version you +have built, not just the current one. In our example so far, `Version` is set +to 10. + +With the `ister` tool configured, build the image with with the following command: + +.. code-block:: bash + + sudo mixer build image --format 1 + +This command outputs an image bootable as a virtual machine and which can be +installed on bare metal. + +Mixer automatically looks for the :file:`release-image-config.json` file but +you can freely choose the filename. To use a different name, simply pass the +:option:`--template` flag when creating your image, for example: + +.. code-block:: bash + + sudo mixer build image --format 1 --template path/to/file.config + +By default, `ister` uses the format version of the build machine it runs on. +Therefore, if the format you are building differs from the format of the |CL| +OS you are building on, you must use the :option:`--format ` +flag. Find the current format version of your OS with the following command: + +.. code-block:: bash + + sudo cat /usr/share/defaults/swupd/format + +Update the next mix version information +*************************************** + +Increment the mix version number for the next mix with the following command: + +.. code-block:: bash + + sudo mixer versions update + +This command automatically updates the mix version stored in the +:file:`mixversion` file incrementing it by 10. To increment by a different +amount, use the :option:`--increment` flag, for example: + +.. code-block:: bash + + sudo mixer versions update --increment 100 + +Alternatively, to set the mix version to a specific value, use the +:option:`--mix-version` flag, for example: + +.. code-block:: bash + + sudo mixer versions update --mix-version 200 + +The :command:`mixer versions update` command does not allow you to set the mix +version to a value lower than its current value. The mix version is expected +to always increase, even if the new mix is undoing an earlier change. + +If you have been tracking your workspace with Git, you can restore the mix to +an earlier state but be careful of "rewriting history" if you are publishing +the mix content to users already. + +Update the upstream version of |CL| used as a base for the mix, with the +following command using the :option:`--upstream-version` flag: + +.. code-block:: bash + + sudo mixer versions update --upstream-version 21070 + +This command also accepts the keyword "latest": + +.. code-block:: bash + + sudo mixer versions update --upstream-version latest + +This command sets the upstream version to the latest released version of +upstream |CL| within the same format version. The :command:`mixer +versions update` command does not allow to set an upstream version to a +value crossing an upstream format boundary. Such values require a +"format bump" build, which is currently a manual process. See :ref:`mixer-format` +for more information. + +Learn which mix version or upstream version you currently are on with the +following command: + +.. code-block:: bash + + sudo mixer versions + +At this point, you can continue to iterate through the workflows and make +modifications as needed. For example: + +#. Add, remove, or modify bundles. +#. Build the chroots with: .. code-block:: bash - # mixer bundle list --tree + sudo mixer build chroots - This will print a tree view of your Mix Bundle List, visually showing what - each bundle includes. - - Bundles fall into two categories: **upstream** and **local**. Upstream - bundles are those provided by Clear Linux. Local bundles are bundles you've - created yourself, or edited versions of upstream bundles. - - Upstream bundle definition files are downloaded and cached for you - automatically by mixer, and are stored in the :file:`upstream-bundles` - directory created in your working directory. Do *not* modify things in this - directory; it is simply a mirror for the tool to use. The tool automatically - caches the bundles for your configured version of Clear Linux (in your - :file:`upstreamversion` file), and cleans up old versions once they are no - longer needed. You can see what upstream bundles are available by running: +#. Build and update with: .. code-block:: bash - # mixer bundle list upstream + sudo mixer build update - Local bundle definition files live in the :file:`local-bundles` directory. - The location of this directory is specified by ``LOCAL_BUNDLE_DIR`` in your - :file:`builder.conf`. For this example, this is - :file:`/home/clr/mix/local-bundles`. You can see what local bundles are - available by running: +#. Optionally,create delta packs with: .. code-block:: bash - # mixer bundle list local + sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update - With either of the above commands, you can pass the :option:`--tree` flag to - see a tree view of what other bundles each bundle includes. +.. _mixer-format: - When looking for a bundle definition file, **mixer always checks local - bundles first, then upstream**. As such, bundles found in - :file:`local-bundles` will always take precedence to upstream bundles of the - same name. This is how "editing" an upstream bundle works; the local, edited - version overrides the version found upstream. (More on editing bundles in - a moment.) - - You can easily **add bundles** to your mix by running: - - .. code-block:: bash - - # mixer bundle add bundle1 [bundle2...] - - This command adds the bundles you specify to your Mix Bundle List - (:file:`mixbundles`). For each bundle you add, mixer checks your local and - upstream bundles to make sure that the bundle you're adding actually exists. - If any cannot be found, an error will be reported. When mixer adds a bundle, - it will tell you whether it was found in local or upstream. You can also see - this information when you run :command:`mixer bundle list`. - - To **remove a bundle** from your mix, run: - - .. code-block:: bash - - # mixer bundle remove bundle1 [bundle2...] - - This command will remove the bundles you specify from your Mix Bundle List - (:file:`mixbundles`). By default, it does not remove the bundle definition - file from your local bundles. If you would like to completely remove a - bundle, including its local bundle definition file, the :option:`--local` - flag can be passed. By default, removing a local bundle file this way will - remove it from your mix as well. If you wish to *only* remove the local - bundle definition file, you can also pass the :option:`--mix=false` flag. - Please note that if you remove a local bundle that was an edited version of - upstream, and that bundle is still in your mix, your mix will now be - referencing the original upstream version of the bundle. If you remove a - bundle that was *only* found locally and still leave the bundle in your Mix - Bundles List, there will no longer be any valid bundle definition file to - refer to, and mixer will produce an error. - - To **edit a bundle definition file**, run: - - .. code-block:: bash - - # mixer bundle edit bundle1 [bundle2...] - - If the bundle is found in your local bundles, mixer will edit this bundle - definition file. If instead the bundle is only found upstream, mixer will - copy the bundle definition file from upstream into your :file:`local-bundles` - directory first. In either case, mixer will launch your default editor to - edit the file. When the editor closes, mixer automatically validates the - edited bundle file, and reports any errors it encounters. If it does find an - error, you have the option of continuing to edit the file as-is, revert and - edit, or skip and keep going to the next bundle. If you skip a file, a backup - of the original file is saved with the ``.orig`` suffix. Because mixer always - checks your local bundles first, edited copies of an upstream bundle will - always take precedence over their upstream counterpart. - - This same command is used to create a totally **new bundle**: if the bundle - name you specify is not found upstream, a blank template is generated in - :file:`local-bundles` with the correct filename. The editor is again launched - for you to fill out the bundle, and validation is performed on exiting. Add - your package name(s) in the bundle definition file to tell it what package(s) - must be installed as part of that bundle. - - Mixer will do basic **validation** on all bundles when they are used - throughout the system: it will check that the bundle syntax is valid and can - be parsed, and that the bundle file has a valid name. If you would like to - manually run this validation on a bundle, you can run: - - .. code-block:: bash - - # mixer bundle validate bundle1 [bundle2...] - - This command has an optional :option:`--strict` flag, which additionally - checks that the rest of the bundle header fields can be parsed and are - non-empty, and that the bundle header ``Title`` field and the bundle filename - match. - - .. note:: - If you initialized your workspace to be tracked as a Git repository - (:command:`mixer init --git`), you may find it useful to apply a git - commit after modifying what bundles are in your Mix Bundle List or - editing a bundle definition file. All of the above :command:`mixer - bundle` commands support an optional :option:`--git` flag that will - automatically apply a git commit when they are finished. - -#. **Build the bundle chroots**. To build all of the ``chroots`` that are - based on the bundles you defined, run the following command in your - workspace: - - .. code-block:: bash - - # sudo mixer build chroots - - If you have many bundles in your mix, this step might take some - time. - - By default, mixer will use the legacy chroot-builder. In this mode, the - bundle definition files for the bundles in your mix will be automatically - gathered into a :file:`mix-bundles` directory in the location specified by - ``BUNDLE_DIR`` in your :file:`builder.conf`. **Do not edit these files**. - Mixer will automatically clear out any contents of this directory before - populating it on-the-fly as chroots are built. - - Mixer now has a new chroot-builder built into the mixer tool itself. While - this is currently an experimental feature, you can (and should) use the new - chroot-builder by passing the :option:`--new-chroots` flag. The legacy - chroot-builder will soon be deprecated, and mixer will use the new version - automatically. - -#. **Create update**. In the workspace, run: - - .. code-block:: bash - - # sudo mixer build update - - When the build completes, you will find your mix update content under - :file:`/home/clr/mix/update/www/VER`. In our example, this will be - located in :file:`/home/clr/mix/update/www/{}`, where - ```` is the mix version you defined (10 by default). - - By default, mixer will use the legacy swupd-server to generate the update - content. Mixer now has a new implementation built into the mixer tool itself. - While this is currently an experimental feature, you can (and should) use the - new swupd-server by passing the :option:`--new-swupd` flag. The legacy - swupd-server will soon be deprecated, and mixer will use the new version - automatically. - - All content to make a fully usable mix will be created by this step, but - note that only *zero packs* are automatically generated. Zero packs are - the content needed to go from nothing to the mix version you just built - content for. To create optional *delta packs*, which allow for - transitioning from one mix version to another, run the pack-maker as - follows: - - .. code-block:: bash - - # sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update - - The pack-maker will generate all delta packs for bundles that have changed - from ``PAST_VERSION`` to ``MIX_VERSION``. If your ``STATE_DIR`` is in a - different location, be sure to specify the location with the ``-S`` - option. For the first build, no delta packs can be created because the - "update" is from version 0. Version 0 implicitly has no content, thus no - deltas can be generated. For subsequent builds, - :file:`mixer-pack-maker.sh` can be run to generate delta content between - them (for example: 10 to 20). - -#. **Creating an image**. Mixer uses the ``ister`` tool to create a bootable - image from your updated content. To configure the image ``ister`` creates, - you will need the ``ister`` config file. You can obtain a default value - from the ``ister`` package: - - .. code-block:: bash - - # cp /usr/share/defaults/ister/ister.json relase-image-config.json - - For reference, you can inspect the ``ister`` config file that `Clear - Linux uses`_ for its releases. - - Note that mixer automatically looks for a file named - :file:`release-image-config.json`, but you can choose whatever name you want. - To use a different name, simply pass the - :option:`--template path/to/file.config` flag when creating your image. - - Edit the config file to include all bundles that you want *preinstalled* - into your image. The rest of the bundles in your mix will be available to your users via: - - .. code-block:: bash - - # swupd bundle add - - Keeping this list small allows for a smaller image size. For a minimal, - base image, this list would be: - - .. code-block:: console - - "Bundles": ["os-core", "os-core-update", "kernel-native"] - - Next, set the ``Version`` field to the mix version content that the image - should be built from. ``ister`` allows you to build an image from any mix - version that you have built, not just the current one. For the first build - example we've been using, ``Version`` would be set to 10. - - Finally, to build the image, run: - - .. code-block:: bash - - # sudo mixer build image --format 1 - - The output from this should be an image that is bootable as a virtual - machine and can be installed on bare metal. - - .. note:: - By default, ``ister`` uses the format version of the build machine it - is running on. Therefore, if the format you are building is different - than the format of the Clear Linux OS that you are building on, you - need to pass :option:`--format `. You can find your - current format version by running: - - .. code-block:: bash - - # cat /usr/share/defaults/swupd/format - -Creating your next mix version -============================== - -**Update the next Mix version info**. To increment your mix version number for -your next mix, run: - - .. code-block:: bash - - # mixer versions update - -This command will automatically update your mix version (stored in the -:file:`mixversion` file), incrementing it by 10. If you'd like to increment by a -different amount, the :option:`--increment` flag can be used: - - .. code-block:: bash - - # mixer versions update --increment 100 - -Alternatively, if you want set your mix version to a specific value, you can -do so with the :option:`--mix-version` flag: - - .. code-block:: bash - - # mixer versions update --mix-version 200 - -Please note that the :command:`mixer versions update` command does not allow you -to set your mix version to something lower than its current value. This is -because your mix version is expected to always increase, even if the new mix is -undoing an earlier change. (If you've been tracking your working directory with -Git, it is possible to restore your mix to an earlier state, but be careful of -"rewriting history" if you are already publishing your mix content to users.) - -From this point you can iterate through the instructions, starting again at -Step 4 and making modifications as needed. For example: - - - Add/remove/modify bundles - - ``sudo mixer build chroots`` - - ``sudo mixer build update`` - - (Optionally) ``sudo mixer-pack-maker.sh --to --from -S /home/clr/mix/update`` - -If you want to update the upstream version of Clear Linux on which your mix is -based, you can do so using the :option:`--upstream-version` flag: - - .. code-block:: bash - - # mixer versions update --upstream-version 21070 - -This command also accepts the keyword "latest": :option:`--upstream-version -latest`. This will set your upstream version to the latest released version of -upstream Clear Linux within the same format version. The :command:`mixer -versions update` command does not allow you to set your upstream version to a -value that would cross an upstream format boundary, as this would require a -"format bump" build, which is currently a manual process. You can read more -about format versions below. - -If you simply wish to learn which mix version or upstream version you currently -are on, you can run: - - .. code-block:: bash - - # mixer versions - - -Format Version +Format version ************** -The ``format`` used in :file:`builder.conf` might be more precisely referred -to as an OS "compatibility epoch". Versions of the OS within a given epoch -are fully compatible with themselves and can update to any version in that -epoch. Across the ``format`` boundary *something* has changed in the OS, -such that updating from build M in format X, to build N in format Y will not -work. Generally this occurs when the software updater or manifests changed -in a way that is no longer compatible with the previous update scheme. +The `Format` variable set in the :file:`builder.conf` can be more precisely +referred to as an OS *compatibility epoch*. Versions of the OS within a given +epoch are fully compatible and can update to any version in that epoch. Across +the `Format` boundary, the OS has changed in such a way, that updating from +build M in format X, to build N in format Y will not work. Generally, this +scenario occurs when the software updater or manifests change in a way no +longer compatible with the previous update scheme. -A format increment is the way we insure pre- and co-requisite changes flow -out with proper ordering. The updated client will only ever update to the -latest release in its respective format version (unless overridden by -command line flags). Thus we can guarantee all clients will update to the -final version in their given format, which *must* contain all the changes -needed to understand the content built in the subsequent format. Only after -reaching the final release in the old format will a client be able to -continue to update to releases in the new format. +Using a format increment, we insure pre- and co-requisite changes flow out +with proper ordering. The updated client only ever updates to the latest +release in its respective format version, unless overridden by command line +flags. Thus, we can guarantee all clients update to the final version in their +given format. The given format *must* contain all the changes needed to +understand the content built in the subsequent format. Only after reaching the +final release in the old format, can a client continue to update to releases +in the new format. -When creating a custom mix, the format version should start at '1', or -some known number, and should increment only when a compatibility breakage is -introduced. Normal updates (for example, updating a software package) do not -require a format increment. +When creating a custom mix, the format version should start at '1' or some +known number. The format version should increment only when a compatibility +breakage is introduced. Normal updates, like updating a software package for +example, do not require a format increment. -.. _Clear Linux update page: https://cdn.download.clearlinux.org/update/ -.. _Format Bumps: https://github.com/clearlinux/swupd-server/wiki/Format-Bumps -.. _Clear Linux uses: https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json +.. _update page: https://cdn.download.clearlinux.org/update/ + +.. _format bumps wiki: https://github.com/clearlinux/swupd-server/wiki/Format-Bumps + +.. _build RPMs instructions: https://github.com/clearlinux/common#build-rpms-for-a-package + +.. _Clear Linux ister configuration file: + https://raw.githubusercontent.com/bryteise/ister/master/release-image-config.json From c2902ec1a5a7336a356c6a64e284155e009f9a19 Mon Sep 17 00:00:00 2001 From: Ecouzens Date: Tue, 6 Mar 2018 13:33:21 -0800 Subject: [PATCH 5/8] copy edit and language cleanup Signed-off-by: Ecouzens --- .../clear-linux/guides/maintenance/mixer.rst | 401 +++++++++--------- 1 file changed, 207 insertions(+), 194 deletions(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index 341f65c0..7178d7ba 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -3,18 +3,18 @@ Use mixer tool ############## -*Mixing* refers to composing an operating system for specific use cases. While -the default |CLOSIA| provides options to install bundles for various server -capabilities, some developers may wish to either augment the operating system -itself with functionality from their own packages or modify the structure of -current bundles to cater to their particular needs. +*Mixing* refers to composing an operating system for specific use cases. +While the default |CLOSIA| provides options to install bundles for various +server capabilities, some developers may wish to either augment the +operating system itself with functionality from their own packages or modify +the structure of current bundles to cater to their particular needs. Prerequisites ************* To start working with the mixer tools, you need a recent image of |CL| with -the `mixer` bundle installed. If the bundle is not yet installed, you can add it -with the :command:`swupd bundle-add` command as follows: +the `mixer` bundle installed. If the bundle is not yet installed, you can +add it with the :command:`swupd bundle-add` command as follows: .. code-block:: bash @@ -56,8 +56,8 @@ these workflows. Create a workspace ****************** -Create an empty directory in your |CL| image to use as a **workspace** for -mixing with the following command: +Use the following command to create an empty directory in your |CL| image to +use as a **workspace** for mixing: .. code-block:: bash @@ -74,14 +74,15 @@ In your workspace, initialize mixer with the following command: sudo mixer init -This command initializes your workspace for you to make a mix at version 10 +This command initializes your workspace so you can make a mix at version 10 based on the latest released upstream |CL| version. If a :file:`builder.conf` -file is not already present in your workspace, mixer creates a default configuration -file along with several version and tracking files and two bundle directories: -:file:`local-bundles` and :file:`upstream-bundles`. +file is not already present in your workspace, mixer creates a default +configuration file, several version and tracking files, and two bundle +directories: :file:`local-bundles` and :file:`upstream-bundles`. If you wish to start with a different version of upstream |CL| or a -different initial mix version, these options can be specified as flags, for example: +different initial mix version, you can specify these options as flags. +For example: .. code-block:: bash @@ -89,29 +90,29 @@ different initial mix version, these options can be specified as flags, for exam Additionally, to build a mix with your own custom RPMs, use the optional - :option:`--local-rpms` flag, for example: + :option:`--local-rpms` flag. For example: .. code-block:: bash sudo mixer init --local-rpms -This command creates the :file:`local-yum` and :file:`local-rpms` directories in your -mix workspace and adds their paths to the generated :file:`builder.conf`. For more -information on using these directories or setting them up manually, see -`Create or locate RPMs for the mix`_. +This command creates the :file:`local-yum` and :file:`local-rpms` +directories in your mix workspace and adds their paths to the generated +:file:`builder.conf`. For more information on using these directories or +setting them up manually, see `Create or locate RPMs for the mix`_. If all upstream |CL| bundles are part of the mix, you can easily add them all -during initialization with the optional :option:`--all-upstream` flag, for +during initialization with the optional :option:`--all-upstream` flag. For example: .. code-block:: bash sudo mixer init --all-upstream -Finally, you may want to track the contents of your mixer workspace with a Git -repository. This is a great way to track changes to your mix's content or to -revert to earlier versions if something goes wrong. Mixer can set this up -automatically with the optional :option:`--git` flag, for example: +Finally, you may want to track the contents of your mixer workspace with a +git repository. This is a great way to track changes to your mix's content +or to revert to earlier versions if something goes wrong. Mixer can set this +up automatically with the optional :option:`--git` flag. For example: .. code-block:: bash @@ -127,13 +128,13 @@ automatically with the optional :option:`--git` flag, for example: Edit builder.conf ***************** -To configure the mixer tool, you must edit the :file:`builder.conf` as needed. +To configure the mixer tool, edit the :file:`builder.conf` as needed. -The file :file:`builder.conf` is read automatically from the current workspace -directory. Use the :option:`--config` flag during initialization to specify a -alternate path to the file as needed. +The file :file:`builder.conf` is read automatically from the current +workspace directory. Use the :option:`--config` flag during initialization +to specify an alternate path to the file as needed. -The :file:`builder.conf` file has different sections, for example: +The :file:`builder.conf` file has different sections. For example: * The `[Builder]` section provides the mixer tools with the required configuration options. This section defines the path where the generated @@ -179,17 +180,19 @@ The following variables require further explanation: * The `LOCAL_BUNDLE_DIR` variable sets the path where mixer stores the local bundle definition files. These bundle definition files include any new, - original bundles you create and edited versions of upstream |CL| bundles. + original bundles you create, along with any edited versions of upstream + |CL| bundles. * The `SERVER_STATE_DIR` variable sets the path for the output of the mix - content. Mixer automatically creates the path for you but the path can be + content. Mixer automatically creates the path for you, but the path can be set to any location. In this example, we use the workspace directory. -* The `BUNDLE_DIR` sets the path where mixer temporarily stores the bundle - definition files while building chroots. Only the legacy chroot-builder uses - this path. By default, mixer does not generate this directory until needed. - In our example, the path is set to :file:`/home/clr/mix/mix-bundles`. The - new chroot-builder does not generate the folder at all. +* The `BUNDLE_DIR` variable sets the path where mixer temporarily stores the + bundle definition files while building chroots. Only the legacy + chroot-builder uses this path. By default, mixer does not generate this + directory until the directory is needed. In our example, the path is set to + :file:`/home/clr/mix/mix-bundles`. The new chroot-builder does not generate + the folder at all. * The `YUM_CONF` variable sets path where mixer automatically generates the :file:`.yum-mix.conf` yum configuration file. The yum configuration file @@ -200,33 +203,32 @@ The following variables require further explanation: certificate file to sign the root :file:`Manifest.MoM` file to provide security for content verification. The value of the `CERT` variable can point to a different certificate. The chroot-builder inserts the - certificate specified here in the + certificate specified in this value into the :file:`/os-core-update/usr/share/clear/update-ca/` path. The software - update client uses this certificate to - verify the :file:`Manifest.MoM` file's signature. For now, we **strongly** - recommend not to modify this line as the certificate `swupd` expects needs - a very specific configuration to sign and verify properly. Mixer - automatically generates the certificate and signs the - :file:`Manifest.MoM` file providing security for your created update - content. + update client uses this certificate to verify the :file:`Manifest.MoM` + file's signature. For now, we **strongly** recommend that you do not + modify this line, as the certificate that `swupd` expects needs to have a + very specific configuration to sign and verify properly. Mixer + automatically generates the certificate and signs the :file:`Manifest.MoM` + file to provide security for the updated content you create. * The `CONTENTURL` and `VERSIONURL` variables set the domain or IP address where mixer serves your updated content and the corresponding version. You - can set the `CONTENTURL` and `VERSIONURL` URL variables to the domain or IP- + can set the `CONTENTURL` and `VERSIONURL` URL variables to the domain or IP address of the update server. The `SERVER_STATE_DIR` directory is hosted here: :file:`/home/clr/mix/update/www`. Creating a symlink to the directory in your server webdir is an easy way to host the content. These URLs are embedded in images created for your mix. The `swupd-client` looks at these - URLs to figure out if there is a new version available and the location from - which to download the updated content. Think of these links as the + URLs to determine both if there is a new version available and the location + from which to download the updated content. Think of these links as the equivalent to the |CL| `update page`_ but for your mix. * The `FORMAT` variable relates to the format bump. To learn more about the - `FORMAT` option, see :ref:`mixer-format` and the `format bumps wiki`_. For - now, leave the `FORMAT` value unchanged. + `FORMAT` option, refer to :ref:`mixer-format` and the `format bumps wiki`_. + For now, leave the `FORMAT` value unchanged. -* The `VERSIONS_PATH` variable set the path for the mix version and upstream - |CL| version two state files: :file:`mixversion` and +* The `VERSIONS_PATH` variable sets the path for the mix version and upstream + |CL| version's two state files: :file:`mixversion` and :file:`upstreamversion`. Mixer creates both files for you when you set up the workspace. @@ -237,18 +239,18 @@ Create or locate RPMs for the mix ********************************* If you create RPMs from scratch, you can use `autospec`, `mock`, `rpmbuild`, -or similar tools to build them. If the RPMs are not built on |CL|, ensure your -configuration and tool-chain builds them correctly for Clear or there is no -guarantee they will be compatible. For more information on building the RPMs -properly visit our `build RPMs instructions`_. +or similar tools to build them. If the RPMs are not built on |CL|, ensure +your configuration and toolchain builds them correctly for |CL|, or else +there is no guarantee they will be compatible. For more information on +building the RPMs properly, refer to our `build RPMs instructions`_. Import RPMs into workspace ************************** -#. Create a :file:`local-rpms` directory in your workspace, for example +#. Create a :file:`local-rpms` directory in your workspace. For example, :file:`/home/clr/mix/local-rpms`. -#. Copy the RPMs into the created directory. +#. Copy the RPMs into the directory you created. #. Add the following line to your :file:`builder.conf` file: @@ -269,25 +271,27 @@ Create a local RPM repo LOCAL_REPO_DIR=/home/clr/mix/local-yum -#. With these values configured, generate the yum repo with the following command: +#. With these values configured, generate the yum repo with the following + command: .. code-block:: bash sudo mixer add-rpms -After the tool exits, you should see the RPMs and a repository data directory in -:file:`/home/clr/mix/local-yum`. If the RPMs are not all in this -:file:`local-yum` directory, check to ensure they are valid RPM files and not -corrupt. +After the tool exits, you should see the RPMs and a repository data +directory in :file:`/home/clr/mix/local-yum`. If the RPMs are not all in this +:file:`local-yum` directory, check to ensure that the RPM files are valid +and not corrupt. List, edit, create, add, remove, or validate bundles **************************************************** The bundles in the mix are specified in the mix bundle list. Mixer stores -this list as a flat file called :file:`mixbundles` in the path set in the -`VERSIONS_PATH` variable of the :file:`builder.conf` file. Mixer generates -the mix bundle list file automatically during initialization. Mixer reads -and writes the bundle list file when you change the bundles of the mix. +this list as a flat file called :file:`mixbundles` in the path set by the +`VERSIONS_PATH` variable of the :file:`builder.conf` file. Mixer +automatically generates the :file:`mixbundles` list file during +initialization. Mixer reads and writes the bundle list file when you change +the bundles of the mix. List the bundles in the mix =========================== @@ -298,54 +302,56 @@ To view the bundles already in the mix, enter the following command: sudo mixer bundle list -The command shows a list of every bundle in the mix. Bundles can include other -bundles and those bundles can themselves include other bundles. When listing -the bundles with this command, mixer automatically recurses through the -includes to show every single bundle in the mix. +This command shows a list of every bundle in the mix. Bundles can include +other bundles, and those nested bundles can themselves include other +bundles. When listing bundles with this command, mixer automatically +recurses through the includes to show every single bundle in the mix. -If you see an unexpected bundle in the list, the bundle probably is included -in another bundle. Use the :option:`--tree` flag to get a better view of how a -bundle ended up in the mix, for example: +If you see an unexpected bundle in the list, that bundle is probably included +in another bundle. Use the :option:`--tree` flag to get a better view of how +a bundle ended up in the mix. For example: .. code-block:: bash sudo mixer bundle list --tree -This command prints a tree view of the mix bundle list explicitly showing each -included bundle. +This command prints a tree view of the mix bundle list that explicitly shows +each included bundle. Bundles fall into two categories: **upstream** and **local**. Upstream bundles are those provided by |CL|. -Mixer automatically downloads and caches upstream bundle definition files. The -definition files are stored in the :file:`upstream-bundles` directory in the -workspace. Do **not** modify the files in this directory. The directory is -simply a mirror for mixer to use. +Mixer automatically downloads and caches upstream bundle definition files. +These definition files are stored in the :file:`upstream-bundles` directory +in the workspace. Do **not** modify the files in this directory. This +directory is simply a mirror for mixer to use. -The mixer tool automatically caches the bundles for the |CL| version configured in the -:file:`upstreamversion` file. Mixer also cleans up old versions once they are no -longer needed. See the available upstream bundles with the following command: +The mixer tool automatically caches the bundles for the |CL| version +configured in the :file:`upstreamversion` file. Mixer also cleans up old +versions once they are no longer needed. You can see the available upstream +bundles with the following command: .. code-block:: bash sudo mixer bundle list upstream -Local bundles are bundles you create or edited versions of upstream bundles. +Local bundles are bundles that you create, or are edited versions of upstream +bundles. Local bundle definition files live in the :file:`local-bundles` directory. The `LOCAL_BUNDLE_DIR` variable sets the path of this directory in your :file:`builder.conf` configuration file. For this example, the path is -:file:`/home/clr/mix/local-bundles`. See the available local bundles with the -following command: +:file:`/home/clr/mix/local-bundles`. You can see the available local bundles +with the following command: .. code-block:: bash sudo mixer bundle list local -Both the local or upstream :command:`bundle list` commands accept the -:option:`--tree` flag to print a tree view explicitly showing each included -bundle. +Both the local and upstream :command:`bundle list` commands accept the +:option:`--tree` flag to print a tree view that explicitly shows each +included bundle. Edit the bundles in the mix =========================== @@ -353,9 +359,9 @@ Edit the bundles in the mix **Mixer always checks local bundles first and the upstream bundles second.** Therefore, bundles in the :file:`local-bundles` directory always take -precedence over the upstream bundles of the same name. +precedence over any upstream bundles that have the same name. -This precedence enables the editing of upstream bundles. The local, edited +This precedence enables you to edit upstream bundles. The local, edited version of the bundle overrides the bundle version found upstream. For example, to edit the `bundle1` definition file, we use the following @@ -365,9 +371,10 @@ command: sudo mixer bundle edit bundle1 -If `bundle1` is found in your local bundles, mixer edits the bundle definition -file. If instead `bundle1` is only found upstream, mixer copies the bundle -definition file from upstream into your :file:`local-bundles` directory first. +If `bundle1` is found in your local bundles, mixer edits the bundle +definition file. If instead `bundle1` is only found upstream, mixer copies +the bundle definition file from upstream into your :file:`local-bundles` +directory first. In both cases, mixer launches your default editor to edit the file. When the editor closes, mixer automatically validates the edited bundle file and @@ -386,7 +393,7 @@ Create bundles for the mix ========================== To create a totally **new bundle**, the bundle name you specify cannot exist -upstream. If that is the case, create a `new-bundle` with the following +upstream. If that is the case, create a :file:`new-bundle` with the following command: .. code-block:: bash @@ -394,14 +401,15 @@ command: sudo mixer bundle edit new-bundle This command generates a blank template in :file:`local-bundles` with the -:file:`new-bundle` filename. Mixer launches the editor for you to fill out the -bundle and performs validation on exiting. Add your package or packages in the -bundle definition file to define the packages to install as part of the bundle. +:file:`new-bundle` filename. Mixer launches the editor for you to fill out +the bundle and performs validation when you exit the editor. Add your package +or packages to the bundle definition file to define the packages to install +as part of the bundle. .. note:: The :command:`mixer bundle edit` accepts multiple bundles at once. Thus, - you can create multiple new bundles in a single command, for example: + you can create multiple new bundles in a single command. For example: .. code-block:: bash @@ -410,19 +418,19 @@ bundle definition file to define the packages to install as part of the bundle. Add bundles to the mix ====================== -Add `bundle1` to your mix easily with the following command: +You can add `bundle1` to your mix with the following command: .. code-block:: bash sudo mixer bundle add bundle1 -This command adds the bundles you specify to your mix bundles list stored in -the :file:`mixbundles` file. For each bundle you add, mixer checks your local -and upstream bundles to ensure the added bundle actually exists. If mixer -cannot find the bundle, it reports back an error. Additionally, when mixer -adds a bundle, it tells you whether the bundle is local or upstream. +This command adds the specified bundles to the mix bundles list stored in +your :file:`mixbundles` file. For each bundle you add, mixer checks your +local and upstream bundles to ensure the added bundles actually exist. If +mixer cannot find the bundle, it reports back an error. Additionally, when +mixer adds a bundle, it tells you whether the bundle is local or upstream. Alternatively, you can learn this information with the -:command:`mixer bundle list` command, see `List the bundles in the mix`_. +:command:`mixer bundle list` command. Refer to `List the bundles in the mix`_. To add multiple bundles at once, use the following command: @@ -439,31 +447,31 @@ Remove `bundle1` from your mix with the following command: sudo mixer bundle remove bundle1 -This command remove the `bundle1` from the mix bundle list stored in the -:file:`mixbundles` file. By default, the command does not remove the bundle +This command removes `bundle1` from the mix bundle list stored in your +:file:`mixbundles` file. By default, this command does not remove the bundle definition file from your local bundles. To completely remove a bundle, -including its local bundle definition file, use the following command with the -:option:`--local` flag: +including its local bundle definition file, use the following command with +the :option:`--local` flag: .. code-block:: bash sudo mixer bundle remove --local bundle1 By default, removing a local bundle file with this command removes the bundle -from the mix as well. To only remove the local bundle definition file, use the -following command with the :option:`--mix=false` flag: +from the mix as well. To only remove the local bundle definition file, use +the following command with the :option:`--mix=false` flag: .. code-block:: bash sudo mixer bundle remove --local --mix=false bundle1 -If you remove a local edited version of an upstream bundle and keep the bundle -in the mix, the mix then references the original upstream version of the -bundle. +If you remove a local, edited version of an upstream bundle and keep the +bundle in the mix, the mix then references the original upstream version of +the bundle. -On the other hand, if you remove a bundle only found locally but keep the -bundle in the mix bundles list, mixer will no longer find a valid bundle -definition file and will produce an error. +On the other hand, if you remove a bundle that is only found locally but +keep the bundle in the mix bundles list, mixer will not find a valid +bundle definition file and will produce an error. Validate the bundles in the mix =============================== @@ -472,17 +480,18 @@ Mixer performs basic **validation** on all bundles when used throughout the system. Mixer checks the validity of the bundle's syntax and name. Mixer also ensures -the bundle can be parsed. Run this validation manually on bundle1 with the +the bundle can be parsed. Run this validation manually on `bundle1` with the following command: .. code-block:: bash sudo mixer bundle validate bundle1 -With the optional :option:`--strict` flag, the command additionally -checks if the rest of the bundle header fields can be parsed, if they are -non-empty, and if the bundle header ``Title`` field and the bundle filename -match. Perform a strict validation of `bundle1` with the following command: +If you use the optional :option:`--strict` flag, the command additionally +checks if the rest of the bundle header fields can be parsed, if the bundle +header fields are non-empty, and if the bundle header ``Title`` field and +the bundle filename match. Perform a strict validation of `bundle1` with the +following command: .. code-block:: bash @@ -497,13 +506,14 @@ Validate multiple bundles with the following command: Managing bundles with Git ========================= -If you initialized your workspace to be tracked as a Git repository -with the :command:`mixer init --git` command, it might be useful to apply a git -commit after modifying the mix bundle list or editing a bundle definition file. +If you initialized your workspace to be tracked as a git repository +with the :command:`mixer init --git` command, it might be useful to apply a +git commit after you modify the mix bundle list or edit a bundle definition +file. -All the :command:`mixer bundle` commands on the previous sections support an -optional :option:`--git` flag. The flag automatically applies a git commit -when the command completes, for example: +All the :command:`mixer bundle` commands in the previous sections support an +optional :option:`--git` flag. This flag automatically applies a git commit +when the command completes. For example: .. code-block:: bash @@ -525,20 +535,21 @@ By default, mixer uses the legacy chroot-builder. In this mode, mixer automatically gathers the bundle definition files for the bundles in the mix into a :file:`mix-bundles` directory. The directory's path is set in the `BUNDLE_DIR` variable in the :file:`builder.conf`. **Do not edit these -files.** Mixer automatically clears out any contents in this directory before -populating it on-the-fly as mixer builds the chroots. +files.** Mixer automatically deletes the contents of the :file:`mix-bundles` +directory before repopulating the directory on-the-fly as mixer builds the +chroots. -We have build a new chroot-builder into the mixer tool itself. While this is -currently an experimental feature, you should use the new chroot-builder. To use -the new chroot-builder, use the following command with the +We have built a new chroot-builder into the mixer tool itself. While this is +currently an experimental feature, you should use the new chroot-builder. To +use the new chroot-builder, use the following command with the :option:`--new-chroots` flag: .. code-block:: bash sudo mixer build chroots --new-chroots -We will soon deprecate the legacy chroot-builder and mixer will use the new version -automatically. +We will soon deprecate the legacy chroot-builder. When we do, mixer will use +the new version automatically. Create an update **************** @@ -552,28 +563,28 @@ Create an update with the following command: When the build completes, you can find the mix update content under :file:`/home/clr/mix/update/www/VER`. In our example, the update content is found in :file:`/home/clr/mix/update/www/{}`. `` -is the mix version defined, 10 by default. +is the defined mix version, which is 10 by default. By default, mixer uses the legacy `swupd-server` to generate the update -content. We have built a new implementation into the mixer tool itself. While -this is currently an experimental feature, you should use the new swupd- -server. To use the the new swupd-server, use the following command with the -:option:`--new-swupd` flag: +content. However, we have built a new implementation into the mixer tool +itself. While this is currently an experimental feature, you should use the +new `swupd-server`. To use the the new `swupd-server`, use the following +command with the :option:`--new-swupd` flag: .. code-block:: bash sudo mixer build update --new-swupd -We will soon deprecate the legacy swupd-server and mixer will use the new version -automatically. +We will soon deprecate the legacy `swupd-server`. When we do, mixer will use +the new version automatically. Mixer creates all the content needed to make a fully usable mix with this step. However, only *zero packs* are automatically generated. Zero packs are the content needed to go from nothing to the mix version for which you just built the content. -Create optional *delta packs*, which allow the transition from one mix version -to another, with the following command: +You can create optional *delta packs*, which allow the transition from one +mix version to another, with the following command: .. code-block:: bash @@ -581,20 +592,20 @@ to another, with the following command: The pack-maker generates all delta packs for the bundles changed from `PAST_VERSION` to `MIX_VERSION`. If your `STATE_DIR` is in a different -location, specify the location with the :option:`-S` flag. Mixer cannot create -delta packs for the first build because the update is from version 0. Version -0 implicitly has no content, thus, mixer can generate no deltas. +location, specify the location with the :option:`-S` flag. Mixer cannot +create delta packs for the first build because the update is from version 0. +Version 0 implicitly has no content. Thus, mixer can generate no deltas. -For subsequent builds, you can run :file:`mixer-pack-maker.sh` to generate -delta content between them, for example: 10 to 20. +For subsequent builds, you can run :file:`mixer-pack-maker.sh` to generate +delta content between them. For example: 10 to 20. Create an image ***************** -Since mixer uses the `ister` tool to create a bootable image from your updated -content, we must first configure the `ister` tool. To configure the image -`ister` creates, we need the `ister` configuration file. Obtain a copy with -the default values from the `ister` package with the following command: +Since mixer uses the `ister` tool to create a bootable image from your +updated content, we must first configure the `ister` tool. To configure the +image `ister` creates, we need the `ister` configuration file. Obtain a copy +with the default values from the `ister` package with the following command: .. code-block:: bash @@ -604,8 +615,8 @@ For reference, you can inspect the `Clear Linux ister configuration file`_ used for releases. Edit the configuration file to include all bundles you want *preinstalled* in -the image. Users can install the bundles in the mix not included in the -configuration file with the following command: +the image. Users can install the bundles in the mix that are not included in +the configuration file with the following command: .. code-block:: bash @@ -618,23 +629,23 @@ smaller image size. For the minimal base image, the list is: "Bundles": ["os-core", "os-core-update", "kernel-native"] -Next, set the `Version` field to the mix version content mixer should use to -build image. `ister` allows you to build an image from any mix version you -have built, not just the current one. In our example so far, `Version` is set -to 10. +Next, set the `Version` field to the mix version that you want the content +mixer to use to build the image. `ister` allows you to build an image from +any mix version that you have built, not just from the current version. In +our example so far, `Version` is set to 10. -With the `ister` tool configured, build the image with with the following command: +With the `ister` tool configured, build the image with the following command: .. code-block:: bash sudo mixer build image --format 1 -This command outputs an image bootable as a virtual machine and which can be -installed on bare metal. +This command outputs an image that is bootable as a virtual machine and +that can be installed on bare metal. -Mixer automatically looks for the :file:`release-image-config.json` file but +Mixer automatically looks for the :file:`release-image-config.json` file, but you can freely choose the filename. To use a different name, simply pass the -:option:`--template` flag when creating your image, for example: +:option:`--template` flag when creating your image. For example: .. code-block:: bash @@ -659,30 +670,31 @@ Increment the mix version number for the next mix with the following command: sudo mixer versions update This command automatically updates the mix version stored in the -:file:`mixversion` file incrementing it by 10. To increment by a different -amount, use the :option:`--increment` flag, for example: +:file:`mixversion` file, incrementing it by 10. To increment by a different +amount, use the :option:`--increment` flag. For example: .. code-block:: bash sudo mixer versions update --increment 100 Alternatively, to set the mix version to a specific value, use the -:option:`--mix-version` flag, for example: +:option:`--mix-version` flag. For example: .. code-block:: bash sudo mixer versions update --mix-version 200 -The :command:`mixer versions update` command does not allow you to set the mix -version to a value lower than its current value. The mix version is expected -to always increase, even if the new mix is undoing an earlier change. +The :command:`mixer versions update` command does not allow you to set the +mix version to a value less than its current value. The mix version is +expected to always increase, even if the new mix is undoing an earlier +change. -If you have been tracking your workspace with Git, you can restore the mix to -an earlier state but be careful of "rewriting history" if you are publishing -the mix content to users already. +If you have been tracking your workspace with git, you can restore the mix to +an earlier state. However, be careful when "rewriting history" if you have +published the mix content to users already. -Update the upstream version of |CL| used as a base for the mix, with the -following command using the :option:`--upstream-version` flag: +Use the following command with the the :option:`--upstream-version` flag to +update the upstream version of |CL| used as a base for the mix: .. code-block:: bash @@ -696,12 +708,12 @@ This command also accepts the keyword "latest": This command sets the upstream version to the latest released version of upstream |CL| within the same format version. The :command:`mixer -versions update` command does not allow to set an upstream version to a -value crossing an upstream format boundary. Such values require a -"format bump" build, which is currently a manual process. See :ref:`mixer-format` -for more information. +versions update` command does not allow you to set an upstream version to a +value that crosses an upstream format boundary. Such values require a +"format bump" build, which is currently a manual process. Refer to +:ref:`mixer-format` for more information. -Learn which mix version or upstream version you currently are on with the +Learn which mix version or upstream version you are currently using with the following command: .. code-block:: bash @@ -724,7 +736,7 @@ modifications as needed. For example: sudo mixer build update -#. Optionally,create delta packs with: +#. Optionally, you can create delta packs with: .. code-block:: bash @@ -735,24 +747,25 @@ modifications as needed. For example: Format version ************** -The `Format` variable set in the :file:`builder.conf` can be more precisely -referred to as an OS *compatibility epoch*. Versions of the OS within a given -epoch are fully compatible and can update to any version in that epoch. Across -the `Format` boundary, the OS has changed in such a way, that updating from -build M in format X, to build N in format Y will not work. Generally, this -scenario occurs when the software updater or manifests change in a way no -longer compatible with the previous update scheme. +The `Format` variable set in the :file:`builder.conf` file can be more +precisely referred to as an OS *compatibility epoch*. Versions of the OS +within a given epoch are fully compatible and can update to any other +version within that epoch. Across the `Format` boundary, the OS has changed +in such a way that updating from build M in format X, to build N in format Y +will not work. Generally, this scenario occurs when the software updater or +software manifests change in a way that they are no longer compatible with +the previous update scheme. Using a format increment, we insure pre- and co-requisite changes flow out with proper ordering. The updated client only ever updates to the latest release in its respective format version, unless overridden by command line -flags. Thus, we can guarantee all clients update to the final version in their -given format. The given format *must* contain all the changes needed to -understand the content built in the subsequent format. Only after reaching the -final release in the old format, can a client continue to update to releases -in the new format. +flags. Thus, we can guarantee that all clients update to the final version +in their given format. The given format *must* contain all the changes +needed to understand the content built in the subsequent format. Only after +reaching the final release in the old format can a client continue to update +to releases in the new format. -When creating a custom mix, the format version should start at '1' or some +When creating a custom mix, the format version should start at `1` or some known number. The format version should increment only when a compatibility breakage is introduced. Normal updates, like updating a software package for example, do not require a format increment. From eb26ba09c66c0b4d8bdf2264c6822ccb540b941b Mon Sep 17 00:00:00 2001 From: Ecouzens Date: Tue, 6 Mar 2018 13:47:17 -0800 Subject: [PATCH 6/8] copy edit and language update Signed-off-by: Ecouzens --- source/clear-linux/guides/maintenance/mixer.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index 7178d7ba..1e66d117 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -765,7 +765,7 @@ needed to understand the content built in the subsequent format. Only after reaching the final release in the old format can a client continue to update to releases in the new format. -When creating a custom mix, the format version should start at `1` or some +When creating a custom mix, the format version should start at "1" or some known number. The format version should increment only when a compatibility breakage is introduced. Normal updates, like updating a software package for example, do not require a format increment. From fdbd1dafc5a11f3f73c6340c4a1acbb46f2e2c9a Mon Sep 17 00:00:00 2001 From: Rodrigo Caballero Date: Wed, 7 Mar 2018 10:45:07 -0600 Subject: [PATCH 7/8] Fix technical inaccuracies introduced by rewording. Additionally, all the unnecessary sudo commands have been removed. Signed-off-by: Rodrigo Caballero --- .../clear-linux/guides/maintenance/mixer.rst | 190 +++++++++--------- 1 file changed, 98 insertions(+), 92 deletions(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index 1e66d117..d66e3f27 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -40,7 +40,7 @@ The following workflow applies if the mix includes your own #. `Create an image`_ Alternatively, the following workflow applies if the mix only uses |CL| -bundles. +content. #. `Create a workspace`_ #. `Generate the starting point for your mix`_ @@ -72,13 +72,14 @@ In your workspace, initialize mixer with the following command: .. code-block:: bash - sudo mixer init + mixer init This command initializes your workspace so you can make a mix at version 10 based on the latest released upstream |CL| version. If a :file:`builder.conf` file is not already present in your workspace, mixer creates a default -configuration file, several version and tracking files, and two bundle -directories: :file:`local-bundles` and :file:`upstream-bundles`. +configuration file. Additionally, mixer creates several version and tracking +files, and two bundle directories: :file:`local-bundles` and +:file:`upstream-bundles`. If you wish to start with a different version of upstream |CL| or a different initial mix version, you can specify these options as flags. @@ -86,44 +87,45 @@ For example: .. code-block:: bash - sudo mixer init --clear-version 21060 --mix-version 100 + mixer init --clear-version 21060 --mix-version 100 Additionally, to build a mix with your own custom RPMs, use the optional - :option:`--local-rpms` flag. For example: +:option:`--local-rpms` flag, for example: .. code-block:: bash - sudo mixer init --local-rpms + mixer init --local-rpms This command creates the :file:`local-yum` and :file:`local-rpms` directories in your mix workspace and adds their paths to the generated -:file:`builder.conf`. For more information on using these directories or +:file:`builder.conf`. If the configuration file already exists, you must add +the paths manually. For more information on using these directories or setting them up manually, see `Create or locate RPMs for the mix`_. -If all upstream |CL| bundles are part of the mix, you can easily add them all +If all upstream |CL| bundles will be part of the mix, you can easily add them all during initialization with the optional :option:`--all-upstream` flag. For example: .. code-block:: bash - sudo mixer init --all-upstream + mixer init --all-upstream Finally, you may want to track the contents of your mixer workspace with a git repository. This is a great way to track changes to your mix's content or to revert to earlier versions if something goes wrong. Mixer can set this -up automatically with the optional :option:`--git` flag. For example: +up automatically with the optional :option:`--git` flag, for example: .. code-block:: bash - sudo mixer init --git + mixer init --git .. note:: You can use any or all of the above optional flags at the same time, for example: .. code-block:: bash - sudo mixer init --clear-version 21060 --mix-version 100 --local-rpms --all-upstream --git + mixer init --clear-version 21060 --mix-version 100 --local-rpms --all-upstream --git Edit builder.conf ***************** @@ -134,7 +136,7 @@ The file :file:`builder.conf` is read automatically from the current workspace directory. Use the :option:`--config` flag during initialization to specify an alternate path to the file as needed. -The :file:`builder.conf` file has different sections. For example: +The :file:`builder.conf` file has different sections, for example: * The `[Builder]` section provides the mixer tools with the required configuration options. This section defines the path where the generated @@ -148,7 +150,7 @@ Edit the configuration file according to your needs with the command: .. code-block:: bash - sudo vim /etc/bundle-chroot-builder/builder.conf + vim /home/clr/mix/builder.conf Your version of the :file:`builder.conf` file should resemble the following example: @@ -194,8 +196,8 @@ The following variables require further explanation: :file:`/home/clr/mix/mix-bundles`. The new chroot-builder does not generate the folder at all. -* The `YUM_CONF` variable sets path where mixer automatically generates the - :file:`.yum-mix.conf` yum configuration file. The yum configuration file +* The `YUM_CONF` variable sets the path where mixer automatically generates + the :file:`.yum-mix.conf` yum configuration file. The yum configuration file points the chroot-builder to the path where the RPMs are stored. * The `CERT` variable sets the path where mixer stores the @@ -204,26 +206,31 @@ The following variables require further explanation: security for content verification. The value of the `CERT` variable can point to a different certificate. The chroot-builder inserts the certificate specified in this value into the - :file:`/os-core-update/usr/share/clear/update-ca/` path. The software - update client uses this certificate to verify the :file:`Manifest.MoM` - file's signature. For now, we **strongly** recommend that you do not - modify this line, as the certificate that `swupd` expects needs to have a - very specific configuration to sign and verify properly. Mixer - automatically generates the certificate and signs the :file:`Manifest.MoM` + :file:`/os-core-update/usr/share/clear/update-ca/` path. The software update + client uses this certificate to verify the :file:`Manifest.MoM` file's + signature. For now, we **strongly** recommend that you do not modify + this line, as the certificate that :abbr:`swupd (Software Updater)` + expects needs to have a very specific configuration to sign and verify + properly. Mixer automatically generates the certificate, if you do not + provide the path to an existing one, and signs the :file:`Manifest.MoM` file to provide security for the updated content you create. * The `CONTENTURL` and `VERSIONURL` variables set the domain or IP address - where mixer serves your updated content and the corresponding version. You - can set the `CONTENTURL` and `VERSIONURL` URL variables to the domain or IP - address of the update server. The `SERVER_STATE_DIR` directory is hosted - here: :file:`/home/clr/mix/update/www`. Creating a symlink to the directory - in your server webdir is an easy way to host the content. These URLs are - embedded in images created for your mix. The `swupd-client` looks at these - URLs to determine both if there is a new version available and the location - from which to download the updated content. Think of these links as the - equivalent to the |CL| `update page`_ but for your mix. + where swupd looks for your update content and the corresponding version. You + must set these variables to the domain or IP-address of the server hosting the + update content. You can use any web server to host your update content. To learn + how to install and configure web server using |CL|, visit + :ref:`web-server-install`. For our example, the web update content within + the `SERVER_STATE_DIR` directory is located here: + :file:`/home/clr/mix/update/www`. If the web server is on same machine as + this directory, you can create a symlink to the directory in your web + server's document root to easily host the content. These URLs are + embedded in images created for your mix. The `swupd-client` looks at + these URLs to determine if a new version is available and the location + from where to download the updated content. These links are equivalent + to the |CL| `update page`_ but for the mix. -* The `FORMAT` variable relates to the format bump. To learn more about the +* The `FORMAT` variable relates to format bumps. To learn more about the `FORMAT` option, refer to :ref:`mixer-format` and the `format bumps wiki`_. For now, leave the `FORMAT` value unchanged. @@ -247,7 +254,7 @@ building the RPMs properly, refer to our `build RPMs instructions`_. Import RPMs into workspace ************************** -#. Create a :file:`local-rpms` directory in your workspace. For example, +#. Create a :file:`local-rpms` directory in your workspace, for example, :file:`/home/clr/mix/local-rpms`. #. Copy the RPMs into the directory you created. @@ -300,23 +307,23 @@ To view the bundles already in the mix, enter the following command: .. code-block:: bash - sudo mixer bundle list + mixer bundle list This command shows a list of every bundle in the mix. Bundles can include -other bundles, and those nested bundles can themselves include other +other bundles. Those nested bundles can themselves include other bundles. When listing bundles with this command, mixer automatically recurses through the includes to show every single bundle in the mix. If you see an unexpected bundle in the list, that bundle is probably included in another bundle. Use the :option:`--tree` flag to get a better view of how -a bundle ended up in the mix. For example: +a bundle ended up in the mix, for example: .. code-block:: bash - sudo mixer bundle list --tree + mixer bundle list --tree -This command prints a tree view of the mix bundle list that explicitly shows -each included bundle. +This command shows a visual representation of the inclusion relationships +between the bundles in the mix. Bundles fall into two categories: **upstream** and **local**. @@ -334,7 +341,7 @@ bundles with the following command: .. code-block:: bash - sudo mixer bundle list upstream + mixer bundle list upstream Local bundles are bundles that you create, or are edited versions of upstream bundles. @@ -347,11 +354,11 @@ with the following command: .. code-block:: bash - sudo mixer bundle list local + mixer bundle list local Both the local and upstream :command:`bundle list` commands accept the -:option:`--tree` flag to print a tree view that explicitly shows each -included bundle. +:option:`--tree` flag to show a visual representation of the inclusion relationships +between the bundles in the mix. Edit the bundles in the mix =========================== @@ -369,17 +376,17 @@ command: .. code-block:: bash - sudo mixer bundle edit bundle1 + mixer bundle edit bundle1 -If `bundle1` is found in your local bundles, mixer edits the bundle +If `bundle1` is found in your local bundles, mixer edits this bundle definition file. If instead `bundle1` is only found upstream, mixer copies the bundle definition file from upstream into your :file:`local-bundles` directory first. In both cases, mixer launches your default editor to edit the file. When the editor closes, mixer automatically validates the edited bundle file and -reports any errors found. If mixer finds an error, you can edit the file as- -is, revert and edit, or skip and move on to the next bundle. If you skip a +reports any errors found. If mixer finds an error, you can edit the file +as-is, revert and edit, or skip and move on to the next bundle. If you skip a file, mixer saves a backup of the original file with the ``.orig`` suffix. Because mixer always checks your local bundles first, edited copies of an upstream bundle always take precedence over their upstream counterpart. You @@ -387,7 +394,7 @@ can edit multiple bundles with the following command: .. code-block:: bash - sudo mixer bundle edit bundle1 bundle2 [bundle3 ...] + mixer bundle edit bundle1 bundle2 [bundle3 ...] Create bundles for the mix ========================== @@ -398,7 +405,7 @@ command: .. code-block:: bash - sudo mixer bundle edit new-bundle + mixer bundle edit new-bundle This command generates a blank template in :file:`local-bundles` with the :file:`new-bundle` filename. Mixer launches the editor for you to fill out @@ -408,21 +415,21 @@ as part of the bundle. .. note:: - The :command:`mixer bundle edit` accepts multiple bundles at once. Thus, - you can create multiple new bundles in a single command. For example: + The :command:`mixer bundle edit` command accepts multiple bundles at once. + Thus, you can create multiple new bundles in a single command, for example: .. code-block:: bash - sudo mixer bundle edit new-bundle1 new-bundle2 [new-bundle3 ...] + mixer bundle edit new-bundle1 new-bundle2 [new-bundle3 ...] Add bundles to the mix ====================== -You can add `bundle1` to your mix with the following command: +Add `bundle1` to your mix with the following command: .. code-block:: bash - sudo mixer bundle add bundle1 + mixer bundle add bundle1 This command adds the specified bundles to the mix bundles list stored in your :file:`mixbundles` file. For each bundle you add, mixer checks your @@ -436,7 +443,7 @@ To add multiple bundles at once, use the following command: .. code-block:: bash - sudo mixer bundle add bundle1 bundle2 [bundle3 ...] + mixer bundle add bundle1 bundle2 [bundle3 ...] Remove bundles from the mix =========================== @@ -445,7 +452,7 @@ Remove `bundle1` from your mix with the following command: .. code-block:: bash - sudo mixer bundle remove bundle1 + mixer bundle remove bundle1 This command removes `bundle1` from the mix bundle list stored in your :file:`mixbundles` file. By default, this command does not remove the bundle @@ -455,7 +462,7 @@ the :option:`--local` flag: .. code-block:: bash - sudo mixer bundle remove --local bundle1 + mixer bundle remove --local bundle1 By default, removing a local bundle file with this command removes the bundle from the mix as well. To only remove the local bundle definition file, use @@ -463,7 +470,7 @@ the following command with the :option:`--mix=false` flag: .. code-block:: bash - sudo mixer bundle remove --local --mix=false bundle1 + mixer bundle remove --local --mix=false bundle1 If you remove a local, edited version of an upstream bundle and keep the bundle in the mix, the mix then references the original upstream version of @@ -476,16 +483,18 @@ bundle definition file and will produce an error. Validate the bundles in the mix =============================== -Mixer performs basic **validation** on all bundles when used -throughout the system. +Mixer performs basic validation on all bundles when used throughout the +system. -Mixer checks the validity of the bundle's syntax and name. Mixer also ensures -the bundle can be parsed. Run this validation manually on `bundle1` with the -following command: +Mixer checks the validity of the bundle's syntax and name. Optionally, you can +run this validation manually on `bundle1` with the following command: .. code-block:: bash - sudo mixer bundle validate bundle1 + mixer bundle validate bundle1 + +.. note:: This command can be useful in many circumstances. One example is + when importing already-existing local bundles from other projects. If you use the optional :option:`--strict` flag, the command additionally checks if the rest of the bundle header fields can be parsed, if the bundle @@ -495,13 +504,13 @@ following command: .. code-block:: bash - sudo mixer bundle validate --strict bundle1 + mixer bundle validate --strict bundle1 Validate multiple bundles with the following command: .. code-block:: bash - sudo mixer bundle validate bundle1 bundle2 [bundle3 ...] + mixer bundle validate bundle1 bundle2 [bundle3 ...] Managing bundles with Git ========================= @@ -513,11 +522,11 @@ file. All the :command:`mixer bundle` commands in the previous sections support an optional :option:`--git` flag. This flag automatically applies a git commit -when the command completes. For example: +when the command completes, for example: .. code-block:: bash - sudo mixer bundle remove --git bundle1 + mixer bundle remove --git bundle1 Build the bundle chroots ************************ @@ -539,7 +548,7 @@ files.** Mixer automatically deletes the contents of the :file:`mix-bundles` directory before repopulating the directory on-the-fly as mixer builds the chroots. -We have built a new chroot-builder into the mixer tool itself. While this is +We have added a new chroot-builder to the mixer tool itself. While this is currently an experimental feature, you should use the new chroot-builder. To use the new chroot-builder, use the following command with the :option:`--new-chroots` flag: @@ -597,7 +606,7 @@ create delta packs for the first build because the update is from version 0. Version 0 implicitly has no content. Thus, mixer can generate no deltas. For subsequent builds, you can run :file:`mixer-pack-maker.sh` to generate -delta content between them. For example: 10 to 20. +delta content between them, for example: 10 to 20. Create an image ***************** @@ -620,7 +629,7 @@ the configuration file with the following command: .. code-block:: bash - sudo swupd bundle add + sudo swupd bundle-add bundle1 Keeping the list of bundles in the configuration file small allows for a smaller image size. For the minimal base image, the list is: @@ -640,12 +649,9 @@ With the `ister` tool configured, build the image with the following command: sudo mixer build image --format 1 -This command outputs an image that is bootable as a virtual machine and -that can be installed on bare metal. - Mixer automatically looks for the :file:`release-image-config.json` file, but you can freely choose the filename. To use a different name, simply pass the -:option:`--template` flag when creating your image. For example: +:option:`--template` flag when creating your image, for example: .. code-block:: bash @@ -667,22 +673,22 @@ Increment the mix version number for the next mix with the following command: .. code-block:: bash - sudo mixer versions update + mixer versions update This command automatically updates the mix version stored in the :file:`mixversion` file, incrementing it by 10. To increment by a different -amount, use the :option:`--increment` flag. For example: +amount, use the :option:`--increment` flag, for example: .. code-block:: bash - sudo mixer versions update --increment 100 + mixer versions update --increment 100 Alternatively, to set the mix version to a specific value, use the -:option:`--mix-version` flag. For example: +:option:`--mix-version` flag, for example: .. code-block:: bash - sudo mixer versions update --mix-version 200 + mixer versions update --mix-version 200 The :command:`mixer versions update` command does not allow you to set the mix version to a value less than its current value. The mix version is @@ -698,30 +704,30 @@ update the upstream version of |CL| used as a base for the mix: .. code-block:: bash - sudo mixer versions update --upstream-version 21070 + mixer versions update --upstream-version 21070 This command also accepts the keyword "latest": .. code-block:: bash - sudo mixer versions update --upstream-version latest + mixer versions update --upstream-version latest This command sets the upstream version to the latest released version of -upstream |CL| within the same format version. The :command:`mixer -versions update` command does not allow you to set an upstream version to a -value that crosses an upstream format boundary. Such values require a -"format bump" build, which is currently a manual process. Refer to -:ref:`mixer-format` for more information. +upstream |CL| within the same format version. The +:command:`mixer versions update` command does not allow you to set an +upstream version to a value that crosses an upstream format boundary. +Such values require a "format bump" build, which is currently a +manual process. Refer to :ref:`mixer-format` for more information. -Learn which mix version or upstream version you are currently using with the -following command: +Optionally, you can learn which mix version or upstream version you are +currently using with the following command: .. code-block:: bash - sudo mixer versions + mixer versions At this point, you can continue to iterate through the workflows and make -modifications as needed. For example: +modifications as needed, for example: #. Add, remove, or modify bundles. #. Build the chroots with: From 5f9bed0e278231d148758c653cc1fbfd34279532 Mon Sep 17 00:00:00 2001 From: "Kevin C. Wells" Date: Wed, 7 Mar 2018 13:22:36 -0800 Subject: [PATCH 8/8] Fix missing word and minor formatting issue --- source/clear-linux/guides/maintenance/mixer.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/source/clear-linux/guides/maintenance/mixer.rst b/source/clear-linux/guides/maintenance/mixer.rst index d66e3f27..90e54dec 100644 --- a/source/clear-linux/guides/maintenance/mixer.rst +++ b/source/clear-linux/guides/maintenance/mixer.rst @@ -222,7 +222,7 @@ The following variables require further explanation: how to install and configure web server using |CL|, visit :ref:`web-server-install`. For our example, the web update content within the `SERVER_STATE_DIR` directory is located here: - :file:`/home/clr/mix/update/www`. If the web server is on same machine as + :file:`/home/clr/mix/update/www`. If the web server is on the same machine as this directory, you can create a symlink to the directory in your web server's document root to easily host the content. These URLs are embedded in images created for your mix. The `swupd-client` looks at @@ -279,7 +279,7 @@ Create a local RPM repo LOCAL_REPO_DIR=/home/clr/mix/local-yum #. With these values configured, generate the yum repo with the following - command: + command: .. code-block:: bash