To avoid misleading info, indicate the correct warning message for
missing state file. Write the details of the default state values as
debug messages.
Fixes#491, #532
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Implement the logging package for mixer.
A common log file can be set for all the mixer commands in the
builder.conf. E.g. `mixer config set Mixer.LOG <filepath>`
The log file and level can also be set for individual mixer
commands using the `--log` and `--log-level` flags respectively.
The various log levels are:
ERROR (1), WARNING (2), INFO (3), DEBUG (4) and VERBOSE (5).
Default log level is 4.
Fixes#666
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
When loading default values during conversion, the filename property was
also overwritten. Since the file should be converted in-place, the
original filename must be preserved.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Currently, the bundle defintions are downloaded from the default
clr-bundles repo https://github.com/clearlinux/clr-bundles/archive/.
The user can now specify a different upstream bundles url using a mixer init flag or mixer
config parameter.
E.g. mixer init --upstream-bundles-url=https://github.com/reaganlo/clr-bundles/archive/
E.g. mixer config set Swupd.UPSTREAM_BUNDLES_URL https://github.com/reaganlo/clr-bundles/archive/
Since there is a tight coupling between the --upstream-version and bundles
repo, the user should ensure that the bundles repo has the same structure as the
clr-bundles repo and the tar.gz files should be named after an upstream
version. i.e. If the mix is based on an upstream-version 31300, the bundles repo
should have a file 31300.tar.gz
Fixes#614
Note: Setting this value using `mixer config set` will only update it in builder.conf.
It does not fetch the upstream bundles. A dedicated command for that purpose
will have to be implemented instead.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
For list values, if defaults are not loaded, they are assumed to be
missing and are not transferred to the converted file.
This patch changes the conversion behavior so that the default values
are always loaded before the config is parsed.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch allows users to configure the compression methods that will
be used for fullfiles creation. The compression method can be set in the
Swupd.COMPRESSION property in builder.conf. Instead of enabling all
methods as previosly, xz is the only method that comes enabled by
default.
Note: internal gzip is still used for Link and Directory fullfile
creation.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch expands mixer config type support to allow int and slice
values to be present in builder.conf. The expandEnv method is now
restricted to string variables and the `config set` command has been
expanded to allow comma separated lists as input for TOML lists.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
MixState needs to know the server state dir in order to find the value
for previous version since its path will not always be relative to the
execution folder.
With this patch, MixState is always loaded after MixConfig. Since the
conversion of older formats of MixConfig account for format transfer,
changing the order does not prevent the format value to be properly
moved to the state file.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The mixer.state file now tracks the previous mix version with the
PREVIOUS_MIX_VERSION field. When this value is not set, it will default
to the LAST_VER or 0 when LAST_VER is invalid.
Format bumps create an important exception for the value of
PREVIOUS_MIX_VERSION. During a format bump, PREVIOUS_MIX_VERSION will be
overridden by the LAST_VER which will prevent PREVIOUS_MIX_VERSION from
altering format bump behavior.
Additionally, the mixer.state version was incremented and tests for
state file version conversions were added.
Signed-off-by: John Akre <john.w.akre@intel.com>
New param added in builder.conf Mixer section to support
customize os-release file.
Example to specify modified os-release file path in Mixer section:
[Mixer]
OS_RELEASE_PATH = "/tmp/modified-os-release"
Note: Either use empty string or do not specify the OS_RELEASE_PATH indicated
Mixer to use default os-release file.
Signed-off-by: sweeaun <swee.aun.khor@intel.com>
One may wish to add or edit repos before building any mixes, but the DNF
(.yum-mix.conf) is not available until build-bundles is run.
Fixes#452
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The existence of local repo and local rpms folder has no impact if they
are not used in the mix. This patch removes the `--local-rpms` flag for
`mixer init` and make it always true. This way both folders are always
created and configured in builder.conf during init.
This patch also update the test cases for config convert to address this
change since.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch changes how Warnings and Error messages are printed
throughout the code to use the `log` package instead of `fmt`. This
allows for finer control over what is printed and where it is printed
to.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
With the automatic conversion on load time, mixer ill always use the new
config format. Even if the mix is initialized with the legacy config, it
will be converted right away. As such, there is no need to allow the
creation of legacy configs anymore nor to check if it is being used.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch introduces automatic conversion of config files to the latest
format. This will seamlessly transition all users to the latest format
without the need for manually converting the config with the `mixer
config convert` command.
This patch also introduces the config_conver.go file, which centralizes
all conversion operations and also the parsing for the legacy config in
order to keep config.go clean and focused on current config features.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The default value for DOCKER_IMAGE_PATH was only set for the old INI
config. As a result, TOML configs would have an empty path that would
generate an invalid docker command to be issued.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
When config is created during mixer init, default values are set for
all variables. To mimic this behavior, LoadDefaults must be called on
conversion as well.
There is also no need to perform a conversion if the parsed version
number is the same as the current version number.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The old INI config does not have versioning, so when performing a
conversion, it needs to be set.
Also, since now the config format is identified on parsing, there is no
need to force Old Config when starting the conversion. This allows the
convert command to also convert between different TOML versions
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Instead of always using '1' as the default FORMAT value, check first if
the value was available in a legacy config. If not, try using the system
format instead. If both fail, fallback to the hardcoded value as it was
previously done.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Use TOML format as the default mixer format. The INI format can still be
used by setting the --new-config flag to false during init.
This change also makes TOML the default format for the bat tests.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Instead of relying on a user provided parameter, try to parse the
builder conf in all formats that mixer understands and assume the format
from there. Currently it can be "TOML with version", "TOML without
version" or "INI".
With this change, once the mix is initialized with 'mixer init', there
is no need to pass the --new-config flag around anymore.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch adds a new comment line on top of new config to specify the
current file format for that config.
With this change, new config files are expected to start with
\#VERSION X.X
where X are integer values.
This patch also change the encoding and decoding of toml to use the
io.Reader and io.Writer interfaces instead of a filename. This allows
the version to be read/write before the toml is processed.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The mixer.state file holds the temporary values used or produced between
mixes or mixes steps. The format variable was the only transient
variable in builder.conf so it was moved to this new state file.
This file should hold all transient values. All the temporary files
should be centralized on this new state file.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch includes the following changes:
Reorder which version builds first to simplify
The +20 build can be built fully at the beginning all the way though,
removing the need to swap tooling to build the +10 (just so things are
built in chronological order). The +10 build is created with the -
possibly - old tooling only after, and reduces the amount of
back-and-forth swapping of tooling.
Fix misc errors with formats and version blocks
The right format number needs to be passed into both functions, and we
*must* ignore the fact that the version and format are lower than the
previous build in the format bump case. It reduces complexity and tool
changes by building the +20 version all the way through first.
Two top level commands now exist for running format bumps:
- build upstream-format This command builds the necessary builds to
cross a mix over a format bump.
- build format-bump This command builds a bump for downstream users.
Fix docker mount path lookup
This patch fixes a bug with the way mixer determined which fields in
'builder.conf' were mountable paths.
Previously, mixer treated every field in [Builder] and [Mixer] as paths
on the filesystem (or paths to files whose parent directories needed to
be mounted). Introducing the "DOCKER_IMAGE_PATH" field broke this
approach.
This patch introduces a new "mount" tag on the config struct fields that
indicates whether a config field represents a mountable path. This has
several benefits:
1) It allows us to know definitively which fields we should look at.
2) It allows us to look at the entire 'builder.conf', not just the
[Builder] and [Mixer] sections.
3) It removes the restriction that paths in 'builder.conf' be absolute.
This absolute check was mostly a sloppy way of checking if a field was
in fact a path.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
Instead of relying on external sources to keep track of the active
config file, store it as a private variable inside MixConfig.
This change prevents inconsistencies regarding filename and also
simplifies the code since the caller doesn't need to initialize the path
or know the filename if it was not the one that initialized that config
object.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch adds a couple new features for LoadDefaults().
First, it introduces LoadDefaultsForPath(), which allows the default
values to be set base on a given path instead of always assuming $PWD.
It also removes the restriction that only allowed defaults to be set
when --new-config set was declared. Removing this restriction ensures
that there are always sane parameters available, even if none exists
in the config file.
Lastly, it moves the check for local RPMs to LoadDefaults from
CreateDefaultConfig(). This prevents GetWd() to be called twice and also
allows LOCAL_REPO_DIR and LOCAL_RPM_DIR to be set in the same call.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch moves config out of the builder package into its own package.
This change is required to reuse MixConfig inside swupd package. Builder
package includes Swupd package, so keeping config there would cause a
cyclic dependency.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
William Douglas