When --offline is passed, any command executed should not reach over the
network. This introduces a tricky scenario when it is passed to a build
command with --native=false (default), because the command in the
container will be run with --offline, but the native mixer binary will
reach over the network to pull/update docker containers before executing
in them. This patch also makes that process offline only, attempting to
use a cached docker image so that it can run offline fully, but exits if
none exist.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The default runtime in upstream Clear will be set to the kata runtime,
which does not support host networking. This is needed for mixer, so we
must force --runtime=runc during all docker runs.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
If the docker image for a given format is updated, users with cached
version of the image may keep running in the older version.
This patch makes sure mixer tries to pull an updated version before
running. If the pull fails, mixer will print a warning instead of
failing to allow users with cached images to keep building offline.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
If DOCKER_IMAGE_PATH is not set, the docker command will be malformed
and the user will receive a cryptic message.
This patch makes sure mixer fails early and provides the user an
informative error.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
When using reflection to iterate the config fields, trying to iterate on
unexported fields will cause mixer to crash. Also, when checking for
docker mounts, there is no need to check any of the unexported fields.
This patch fix this crash by skipping the search on unexported fields.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch includes the following changes:
Reorder which version builds first to simplify
The +20 build can be built fully at the beginning all the way though,
removing the need to swap tooling to build the +10 (just so things are
built in chronological order). The +10 build is created with the -
possibly - old tooling only after, and reduces the amount of
back-and-forth swapping of tooling.
Fix misc errors with formats and version blocks
The right format number needs to be passed into both functions, and we
*must* ignore the fact that the version and format are lower than the
previous build in the format bump case. It reduces complexity and tool
changes by building the +20 version all the way through first.
Two top level commands now exist for running format bumps:
- build upstream-format This command builds the necessary builds to
cross a mix over a format bump.
- build format-bump This command builds a bump for downstream users.
Fix docker mount path lookup
This patch fixes a bug with the way mixer determined which fields in
'builder.conf' were mountable paths.
Previously, mixer treated every field in [Builder] and [Mixer] as paths
on the filesystem (or paths to files whose parent directories needed to
be mounted). Introducing the "DOCKER_IMAGE_PATH" field broke this
approach.
This patch introduces a new "mount" tag on the config struct fields that
indicates whether a config field represents a mountable path. This has
several benefits:
1) It allows us to know definitively which fields we should look at.
2) It allows us to look at the entire 'builder.conf', not just the
[Builder] and [Mixer] sections.
3) It removes the restriction that paths in 'builder.conf' be absolute.
This absolute check was mostly a sloppy way of checking if a field was
in fact a path.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
This patch adds an access check on each directory in the config file
before mounting it to the container. This prevents the user from
accidentally (or surreptitiously) granting the container access to
directories they shouldn't.
The config parsing for directory extraction code is also refactored to
provide better results and more accurate error messages.
This patch also adds unit tests for directory access checks, directory extraction,
and mount path list reduction.
For these tests to not fail inside the Docker container on Travis, they
need to not be run as root. This patch thus also creates a non-root user
(but with wheelnopw membership), and runs as this user instead.
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
This patch Dockerizes mixer build commands. A new '--native' flag is added
that, if passed, will run mixer on the host system the way it always has.
This defaults to false.
For build commands, if '--native' is false, mixer now pulls an image file
published by upstream that contains the core Clear Linux OS and mixer
toolchain for the version of Clear Linux defined in the upstreamversion file,
which mixer than uses to build a Docker image. Mixer then re-runs the original
mixer build command within a container of that Docker image.
This ensures that you are always using a version of the mixer toolchain
compatible with the upstream version off of which you are building. This is
required for building across format boundaries. Additionally, mixer now
checks if a build crosses format boundaries, and instructs you how to
first perform a format-bump build.
Mixer attempts to re-use already downloaded or built Docker images to
minimize the performance impact of running inside a container. This eliminates
the time spent on re-building the container image across subsequent
runs. Mixer uses the latest-released version for a given format, however,
so image content will become stale regularly.
Signed-off-by: Kevin C. Wells <kevin.c.wells@intel.com>
Tudor Marcu