When mixer's -c/--config option was used, the referenced config was read
too late to set a default log location. As a result, the log location
was always set to the empty string (disabling logging), that is, unless
--log VALUE was specified, or if mixer's current working directory
contained a builder.conf with LOG defined.
The root cause of this bug is an order-of-operations issue with cobra
integration; fix the issue by setting the log location in
PersistentPreRunE() instead of init(), ensuring that the command line
arguments have been processed, with `configFile` being set to the
-c/--config value if specified.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Implement the logging package for mixer.
A common log file can be set for all the mixer commands in the
builder.conf. E.g. `mixer config set Mixer.LOG <filepath>`
The log file and level can also be set for individual mixer
commands using the `--log` and `--log-level` flags respectively.
The various log levels are:
ERROR (1), WARNING (2), INFO (3), DEBUG (4) and VERBOSE (5).
Default log level is 4.
Fixes#666
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
Mixer will now always update the LAST_VER file after `build update`.
This file is used by `build validate` (MCA) to determine whether
the version in comparison is a +10.
Fixes#735
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
When a downstream mix crossed multiple upstream format bumps, the
upstreamversion and mix state variables were updated in the workspace,
but the changes were not updated in memory. This change reloads the
upstreamversion when it is staged/unstaged and reloads the mix state
after crossing each format boundary. Additionally, this change
increments the format number for each upstream format that the
downstream mix crosses.
Fixes#688
Signed-off-by: John Akre <john.w.akre@intel.com>
Passing '--local' with mixer bundle create will skip
the upstream check and create a new empty local bundle
if it does not already exist.
fixes#713
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
Existing image and output directories for the current mix version will
be removed when building a mix. This prevents mixer from building on top
of directories contaminated by a prior mix.
Fixes#669
Signed-off-by: John Akre <john.w.akre@intel.com>
The build commands require root in order to run dnf, clr-installer, or
access files in the update folder that are owned by root.
Fixes#690
Signed-off-by: John Akre <john.w.akre@intel.com>
Now the mixer repo list command displays repo priorities and the mixer
repo add command can set the repo's priority with the --priority flag.
Signed-off-by: John Akre <john.w.akre@intel.com>
The set-priority repo command updates the priority value for the
specified repo in the DNF config file. This command uses the following
format:
mixer repo set-priority <repo> <priority>
Fixes#691
Signed-off-by: John Akre <john.w.akre@intel.com>
The MCA statistics table replaced the tabwriter implementation with the
tablewriter package. Now the statistics table is a complete box with
line wrapping support for long lines.
This change also adds the --table-width parameter which is used to set the
maximum width of the MCA statistics table. The default value is the
width of the calling terminal and the table can be disabled by providing
a negative number for the table-width.
Signed-off-by: John Akre <john.w.akre@intel.com>
Currently, for repos with "file" baseurl scheme, mixer expects the
rpms to be directly within the baseurl path.
But it is possible that rpms can be in sub directories within the repo baseurl.
In order to handle this, if an rpm is not found directly within the repo baseurl,
run repoquery to determine the actual location of the rpm.
This is expensive and it is recommended that rpms are stored directly
within the repo baseurl.
Includes a bat test.
Fixes#674
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Currently mixer allows users to set multiple repos using 'mixer repo set' commands and name them anyhow.
But the code within the mixer is tightly coupled to having only two repos - "clear" and "local".
This inconsistency is fixed.
fixes#654, #659
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
Currently, `mixer repo remove` displays incorrect messages if repo does
not exist.
This fix displays the correct message.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Switch from ister.py to clr-installer for generating OS images.
Will migrate existing ister JSON file if it exist.
Both ISO generate and Raw Image retention enabled by default.
Signed-off-by: Mark D Horn <mark.d.horn@intel.com>
Built-in Docker support was originally added to enable auto-format bump.
At that time, mixer could only handle one format version at a time, so
we needed 2 different binaries to cross a format boundary: 1 for the +10
and one for the +20. The docker support would allow us to download an
image with the right version for that given format and simplify the
process.
Mixer now has support for multiple formats and a single binary can
handle the whole format bump process. Built in docker image is no longer
needed.
Note: For Docker usage due to compatibility issues with the host system,
mixer-ci Docker image should be used instead.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Format mismatch between the host and upstream was an issue when a given
mixer version could only generate a single format. Since we introduced
multi-format capability, the format of the host doesn't influence the
build anymore and this check can be safely ignored.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Currently, the bundle defintions are downloaded from the default
clr-bundles repo https://github.com/clearlinux/clr-bundles/archive/.
The user can now specify a different upstream bundles url using a mixer init flag or mixer
config parameter.
E.g. mixer init --upstream-bundles-url=https://github.com/reaganlo/clr-bundles/archive/
E.g. mixer config set Swupd.UPSTREAM_BUNDLES_URL https://github.com/reaganlo/clr-bundles/archive/
Since there is a tight coupling between the --upstream-version and bundles
repo, the user should ensure that the bundles repo has the same structure as the
clr-bundles repo and the tar.gz files should be named after an upstream
version. i.e. If the mix is based on an upstream-version 31300, the bundles repo
should have a file 31300.tar.gz
Fixes#614
Note: Setting this value using `mixer config set` will only update it in builder.conf.
It does not fetch the upstream bundles. A dedicated command for that purpose
will have to be implemented instead.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Extracting rpm uisng rpm2archive is faster than dnf and will
improve performace significantly.
Fixes#631
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
Mixer upstream-format should run with optional flag --new-format. If
--new-format is not set, the default new format should be current
format+1. This default behaviour was not getting set for the current mixer version.
Fixed the behaviour to reflect the default behaviour.
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
golangci-lint identified a couple errors that were not found by
gometalinter. This patch fixes the detected errors.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The os-core-update-index manifest was unused and injected into the mix
by mixer. This change removes support for the index manifest in formats
larger than 28 and deletes the index manifest when a mix passes format
28.
Additionally, this change consolidates the index manifest implementation
into the swupd package so that it can more easily be enabled or disabled.
Signed-off-by: John Akre <john.w.akre@intel.com>
In some cases a single DNF config file is not sufficient to correctly
download packages for the mix versions specified by the from and to
flags. The --from-repo-url and --to-repo-url flags were added to
temporarily override the baseurl value for the specified repo in the DNF
config when downloading packages for the from and to versions respectively.
Multiple instances of these flags can be used to override additional
repo/baseurl pairs. An example flag format is shown below:
--from-repo-url <repo>=<URL>
--to-repo-url <repo>=<URL>
Signed-off-by: John Akre <john.w.akre@intel.com>
MixState needs to know the server state dir in order to find the value
for previous version since its path will not always be relative to the
execution folder.
With this patch, MixState is always loaded after MixConfig. Since the
conversion of older formats of MixConfig account for format transfer,
changing the order does not prevent the format value to be properly
moved to the state file.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
When doing manual format bumps, the upstream version for +20 needs to be
updated to the upstream version in the new format, but since the format
differs, the build will fail.
This PR adds a new flag to `mixer versions update` and to `mixer build`
subcommands that allows this check to be skipped.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The `--clear-version` flag in mixer init accepts a string as parameter
to allow the user to set it to `latest`, which will auto fetch the
upstream version. For all other cases, this flag must be provided an
integer number. This patch clarifies the valid values for this flag both
in the error message and in the help description.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The mixer.state file now tracks the previous mix version with the
PREVIOUS_MIX_VERSION field. When this value is not set, it will default
to the LAST_VER or 0 when LAST_VER is invalid.
Format bumps create an important exception for the value of
PREVIOUS_MIX_VERSION. During a format bump, PREVIOUS_MIX_VERSION will be
overridden by the LAST_VER which will prevent PREVIOUS_MIX_VERSION from
altering format bump behavior.
Additionally, the mixer.state version was incremented and tests for
state file version conversions were added.
Signed-off-by: John Akre <john.w.akre@intel.com>
MCA compares two versions to validate that the manifest file changes
align with corresponding package changes. All manifest file changes (except
for a few special cases) will be compared against resolved package file
changes. Any mismatches will be reported as errors. When there are no
errors, package statistics will be displayed.
The following command runs the MCA test:
mixer build validate --from <version> --to <version>
Signed-off-by: John Akre <john.w.akre@intel.com>
swupd-server used to support building bsdiff based deltas of manifest
files. This change adds that ability back for bundle manifests with
the 'mixer build delta-manifests' command.
Download failures can occur when downloading large sets of RPMs from a
server with a poor connection. This change adds retry functionality to
attempt RPM downloads multiple times. Also, the retries flag was added to
the following Mixer build commands.
retries:
Fixes: #531
Signed-off-by: John Akre <john.w.akre@intel.com>
The clean flag for the mixer build all command was not exposed to users
to change from the default value. Now the clean flag can be set for the
mixer build all command.
Signed-off-by: John Akre <john.w.akre@intel.com>
Use cdn instead of download.clearlinux.org as the standard upstream URL,
because it's more reliable. Other usages of download.clearlinux.org were
also changed with the exception of version number downloads. Downloading
version numbers from download.clearlinux.org is still preferred because
we don't need to be concerned with caching.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
If the --clear-version flag is not used during init, clear version will
default to "latest" which causes build to fetch upstream for the latest
version despite the value of the offline flag. This change makes mixer
fail early and inform the user of the missing parameter on init.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Earlier the new-swupd and new-config flags were incorrectly marked for
deprecation. After correctly deprecating these flags, they are no longer
present in the help message and print a deprecation message when used.
Signed-off-by: John Akre <john.w.akre@intel.com>
With the removal of the editor feature in `bundle edit`,
the term `edit` is not relevant anymore. Hence, changing it to `create`.
In order to prevent breakage of existing use-cases,
`edit` is aliased to `create`and `--suppress-editor` is made hidden and deprecated.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
`bundle edit` will no longer open the editor. It will just create new bundles or copy existing bundles.
This command will locate the bundle by first looking in local-bundles, and then in upstream-bundles.
If the bundle is only found upstream, the bundle file will be copied to the local-bundles directory.
If the bundle is not found anywhere, a blank template will be created with the correct name.
This commit also fixes issue #448 and increases test coverage for `bundle validate`.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
With the exception of init, all other subcommands have their own file.
This patch moves init to a separate file to keep it consistent with the
rest of the code.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The root preRun() function accumulated a lot of responsibility over time
that are better located into sub command. This PR deconstructs that
function in order to have a simpler codepath.
Now, the prerun performs only 3 actions:
- Start profiling if needed.
- Check if it is the root command.
- Check if the command should run on a container or not.
The container logic has also been simplified. All commands are not
treated as native commands. During the pre-run, if a command has the
container marker, the current execution is aborted and the command is
re-executed inside a container. This way, the native flag only needs to
be checked once during execution.
Most of the logic in the root preRun() were exclusive to the `build`
subcommand, so they were moved to the build preRun().
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The --config flag was added to every single subcommand in mixer. Instead
of doing so, make it a global flag.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
When mixer is executed without any argument, it should print its usage
message and exit. In order to do so, mixer must skip all it's checks on
the pre-run and just return if the current command is the top level
command alone.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Users basing their mixes on upstream Clear and not acting as pure OSV's
need to have the correct mixes automatically generated for them. This
creates the two mixes as needed and then resets their upstreamversion to
the version they were attempting to build.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Now the 'mixer build all' command supports the --increment flag. When the
--increment flag is passed to 'mixer build all', the mixversion will be
incremented. Otherwise, the mixversion will remain unchanged. This fixes
an issue where delta packs could not be created after using 'mixer build all'.
Fixes#445
Signed-off-by: John Akre <john.w.akre@intel.com>
The existence of local repo and local rpms folder has no impact if they
are not used in the mix. This patch removes the `--local-rpms` flag for
`mixer init` and make it always true. This way both folders are always
created and configured in builder.conf during init.
This patch also update the test cases for config convert to address this
change since.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
William Douglas