Implement the logging package for mixer.
A common log file can be set for all the mixer commands in the
builder.conf. E.g. `mixer config set Mixer.LOG <filepath>`
The log file and level can also be set for individual mixer
commands using the `--log` and `--log-level` flags respectively.
The various log levels are:
ERROR (1), WARNING (2), INFO (3), DEBUG (4) and VERBOSE (5).
Default log level is 4.
Fixes#666
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
When a downstream mix crossed multiple upstream format bumps, the
upstreamversion and mix state variables were updated in the workspace,
but the changes were not updated in memory. This change reloads the
upstreamversion when it is staged/unstaged and reloads the mix state
after crossing each format boundary. Additionally, this change
increments the format number for each upstream format that the
downstream mix crosses.
Fixes#688
Signed-off-by: John Akre <john.w.akre@intel.com>
Format bumps require additional special case file handling for the
os-core and os-core-update bundles. Error messages are generated when
any special case files are missing and warning messages are generated
when they may be missing. The warning messages are necessary because
there currently is not enough information to determine the +20 version
which has file exceptions. Also, this change adds bundle deletion
support.
Signed-off-by: John Akre <john.w.akre@intel.com>
MCA compares two versions to validate that the manifest file changes
align with corresponding package changes. All manifest file changes (except
for a few special cases) will be compared against resolved package file
changes. Any mismatches will be reported as errors. When there are no
errors, package statistics will be displayed.
The following command runs the MCA test:
mixer build validate --from <version> --to <version>
Signed-off-by: John Akre <john.w.akre@intel.com>
There is no information available in offline mode to determine if a
format bump is required. This change also allows to manually set the
value for upstream version without directly editing the file.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The mixer.state file holds the temporary values used or produced between
mixes or mixes steps. The format variable was the only transient
variable in builder.conf so it was moved to this new state file.
This file should hold all transient values. All the temporary files
should be centralized on this new state file.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Apparently there was an error during rebase of the format bump patch
that brought back BuildConf variable which is not supposed to exist
anymore. This reverts it back to the proper state.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The only use case where a missing LAST_VER is a valid use case is when
checking if a bump is needed since the user might be running the build
for the first time. In all other cases, the caller expects a valid
version to be returned is error is nil. This caused mixer to print some
cryptic errors if those other callers were used on a new workspace.
This patch changes the default behavior to always return an error if
LAST_VER is missing and let the caller decide using os.IsNotExist() api
if the error should be propagated or not.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The SetProperty API is used to search for the property withing config,
set it and save the config. In this code, the property is set directly
just a few lines above, so saving is already enough to propagate it to
the config file.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Fix a number of AFB bugs:
- Previous build's upstream format was being incorrectly looked up,
resulting in upstream format boundary crossing detection failing.
- PreRun check was incorrectly skipping format build commands, resulting
in "stage for mix" being run twice.
- Only the UInt32 upstream version was being modified, resulting in the
second half of 'mixer build upstream-format' being run in the wrong
container.
- The wrong format was being printed out for 'mixer versions' if
crossing an upstream format boundary.
Refactor Cobra command PreRun
Simplified the PreRun logic that happens before every command. This also
fixes an issue where format bump build commands could be run with
--native=false, when they need to be run natively.
Fix AFB mixversion and LAST_VER setting
1) Changes how the format bump builds set mixversion so that it no
longer skips a version number in some situations.
2) Updates the update/image/LAST_VER file after the +10 build back
to the +20 build value, so mixer is ready to do the next mix.
Refactor file download code
This patch refactors and consolidates the different file download code
scattered throughout helper.go and builder.go.
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This patch includes the following changes:
Reorder which version builds first to simplify
The +20 build can be built fully at the beginning all the way though,
removing the need to swap tooling to build the +10 (just so things are
built in chronological order). The +10 build is created with the -
possibly - old tooling only after, and reduces the amount of
back-and-forth swapping of tooling.
Fix misc errors with formats and version blocks
The right format number needs to be passed into both functions, and we
*must* ignore the fact that the version and format are lower than the
previous build in the format bump case. It reduces complexity and tool
changes by building the +20 version all the way through first.
Two top level commands now exist for running format bumps:
- build upstream-format This command builds the necessary builds to
cross a mix over a format bump.
- build format-bump This command builds a bump for downstream users.
Fix docker mount path lookup
This patch fixes a bug with the way mixer determined which fields in
'builder.conf' were mountable paths.
Previously, mixer treated every field in [Builder] and [Mixer] as paths
on the filesystem (or paths to files whose parent directories needed to
be mounted). Introducing the "DOCKER_IMAGE_PATH" field broke this
approach.
This patch introduces a new "mount" tag on the config struct fields that
indicates whether a config field represents a mountable path. This has
several benefits:
1) It allows us to know definitively which fields we should look at.
2) It allows us to look at the entire 'builder.conf', not just the
[Builder] and [Mixer] sections.
3) It removes the restriction that paths in 'builder.conf' be absolute.
This absolute check was mostly a sloppy way of checking if a field was
in fact a path.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
This step cannot occur for every format bump build as it previously did,
as it is only valid for format bump builds for crossing an upstream
format boundary. If you are simply bumping your own mix format,
independent of upstream, this should not occur.
Moved the call to CheckFormatBumpNeeded, as this is the only place that
definitively knows this step needs to occur.
Adds an "UnstageMixFromBump" command that resets the "upstreamversion"
file back to the contents found in "upstreamversion.bump", if it exists.
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
This patch Dockerizes mixer build commands. A new '--native' flag is added
that, if passed, will run mixer on the host system the way it always has.
This defaults to false.
For build commands, if '--native' is false, mixer now pulls an image file
published by upstream that contains the core Clear Linux OS and mixer
toolchain for the version of Clear Linux defined in the upstreamversion file,
which mixer than uses to build a Docker image. Mixer then re-runs the original
mixer build command within a container of that Docker image.
This ensures that you are always using a version of the mixer toolchain
compatible with the upstream version off of which you are building. This is
required for building across format boundaries. Additionally, mixer now
checks if a build crosses format boundaries, and instructs you how to
first perform a format-bump build.
Mixer attempts to re-use already downloaded or built Docker images to
minimize the performance impact of running inside a container. This eliminates
the time spent on re-building the container image across subsequent
runs. Mixer uses the latest-released version for a given format, however,
so image content will become stale regularly.
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
This adds support for detecting if a format bump is needed, and the
supplementary functions to perform one. All of the build steps are done
for the user, and just need to be called in the correct order to
generate the two builds.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Ashlesha Atrey