This release includes support for building 3rd party content along with
other enhancements and bug fixes.
New features (beta) for 3rd party support:
- Create a minimal mix using just an os-core bundle that can be empty.
- Add BUILD_ID field in os-release file that specifies the upstream
version from which the mix is created.
- Build bundles from contents by providing the content path using a new
keyword "content()" in the bundle definition.
- Introduce a new export flag "x" in bundle manifests to indicate exportable
files in "/bin/", "/usr/bin/" and "/usr/local/bin/".
The user can override this flag by specifying the files using a new
keyword "un-export()" in the bundle definition.
Other changes:
- Fix error to handle bundles that contain only optional bundles.
- Fix build errors due to stale dnf cache by cleaning the dnf cache before using it.
- Clean mix workspace before building bundles.
- Add --local flag for bundle creation that skips the upstream check and
creates empty local bundles.
- Improve performance by writing mixbundles file just once for the resolved bundle set.
- Make bundle messages consistent.
- Print delta pack results together to prevent interleaved results.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
When using content chroots, file uid/gid values should be preserved when
copying them to the full chroot.
Fixes#727
Signed-off-by: John Akre <john.w.akre@intel.com>
When resolving the full mix bundle set, the mixbundles file is now
updated once with the finalized set of bundles. Previously, the
mixbundles file was written many times with intermediate results which
scaled poorly when building with many bundles.
Fixes#724
Signed-off-by: John Akre <john.w.akre@intel.com>
When building a mix with a version of a package repo that contains
existing cache contents, dnf may resolve incorrect results.
For instance, Mixer does not version the local package repo, so when
incrementally building a mix with local content, dnf will continue to
re-use the same cache for the local repo. This was problematic when the
version of an rpm in the local repo was updated. In this case, the stale
cache metadata was used to incorrectly resolve files for the package.
Cleaning the dnf cache before using it prevents old cache metadata from
impacting dnf results.
Fixes#719
Signed-off-by: John Akre <john.w.akre@intel.com>
By default, files with the prefix "/bin/", "/usr/bin/" and "/usr/local/bin/"
will have a new export flag "x" in the 4th byte of the flag section of the
bundle Manifest.
The user can override this by using a new keyword "un-export()" in
the bundle definition file.
E.g. un-export(/usr/bin/file2)
Note: The export flag is at a bundle-file level and hence does not get
written to the Full manifest.
Fixes#708
Includes bat tests for the export flag feature.
Also, adds an entry for the content-chroot tests to Travis.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Change RenameFlag to MiscFlag since "rename" has been deprecated and
this flag will be used as a placeholder for various other flags.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Currently, build bundles generates an error if one of the mix bundles
has only an 'also-add()' entry in its definition.
Instead, build bundles should not error out and must process such
bundles.
Fixes#718
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Delta packs are generated concurrently, so their results output must be
printed once to prevent interleaved results. Additionally, this change
redirects warnings to stderr so that can be separated from the results.
Fixes#551
Signed-off-by: John Akre <john.w.akre@intel.com>
Passing '--local' with mixer bundle create will skip
the upstream check and create a new empty local bundle
if it does not already exist.
fixes#713
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
Bundle chroots do not resolve conflicts against the full chroot and the
workflow is not streamlined. Now, the bundle definition file should be
used to integrate custom content into the mix.
The bundle definition file supports content chroots which are integrated
into the bundle's info file and resolved against the full chroot.
Ex: content(/path)
Signed-off-by: John Akre <john.w.akre@intel.com>
Files in the content chroots specified by the "contents" keyword in the
bundle definition file are compared against the full chroot to resolve
conflicts which are fatal. When there are no conflicts, files from the
content chroots are copied into the full chroot and added to the
corresponding bundle-info file.
Fixes#705
Signed-off-by: John Akre <john.w.akre@intel.com>
The CopyFileWithOptions function has the ability to disable symlink
resolution, preserve the source file permissions, and disable disk syncs.
Signed-off-by: John Akre <john.w.akre@intel.com>
The content keyword in the bundle definition file specifies the path to
a chroot whose file contents will be added to the bundle's manifest.
Ex: content(/path)
Signed-off-by: John Akre <john.w.akre@intel.com>
Existing image and output directories for the current mix version will
be removed when building a mix. This prevents mixer from building on top
of directories contaminated by a prior mix.
Fixes#669
Signed-off-by: John Akre <john.w.akre@intel.com>
If mix is minimal mix with empty os-core, directory path for
os-release file does not exist.
Create a directory path before updating the os-release file
BUILD_ID field in os-release file specifies the upstream Clear
Linux version from which the mix is being created.
If upstream Clear Linux is not found, BUILD_ID is empty.
fixes#706
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
In order to enable a minimal mix for 3rd party, "filesystem" package is made optional.
In case the "filesystem" package is found in any of the mix bundles,
install the "filesystem" package first. If it’s not found in any of the mix bundles,
then do not install it.
swupd-client expects a mix to have a bundle named "os-core" and the "os-release" file.
Hence they are a hard requirement for creating a mix in Mixer.
If the "os-release" file is not found, then create and inject a default one.
fixes#704
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
This release includes minor enhancements and bug fixes.
Changes:
- Add root permission check to inform the user when certain commands
require root permission.
- Refactor error handling in go channels to prevent potential hanging of
the application in certain scenarios.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
The build commands require root in order to run dnf, clr-installer, or
access files in the update folder that are owned by root.
Fixes#690
Signed-off-by: John Akre <john.w.akre@intel.com>
Currently while catching errors using channels and select
statements, when the error is caught in select statements, it
is not breaking from the outside loop.
The correct approach is to let the existing go routines finish
executing and return one of the errors caught by the error channel.
It should also close the channel in order to stop executing new go
routines.
fixes#673
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
This release includes transitioning to `go mod` to manage dependencies along with
minor enhancements and bug fixes.
Changes:
- vendor: Use `go mod` to manage dependencies.
- Makefile: Remove unused local gopath and set default gopath.
- Makefile: Install go linters if it's not present.
- MCA: Refactor diff results analysis section to make it easier to handle
special case manifest files.
- MCA: Add support for bundle tracking file.
- swupd: Retain previously deleted file entries when the previous manifest's
filelist is longer than the new manifest's list.
- repo: Add the option to set a repo’s priority using the `set-priority` repo command.
- Detect deprecated bundles when crossing an upstream format bump in a downstream mix.
- When packages cannot be resolved during `build bundles`, indicate the unresolved
packages in the error message.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
If `build bundles` fail to resolve packages,
print the missing packages in the error message.
Fixes#697
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
Validate that upstream and local bundles are correctly deleted when
crossing an upstream format as a downstream mix.
Signed-off-by: John Akre <john.w.akre@intel.com>
When searching for deprecated bundles to detect bundle deletion, both
the upstream and downstream bundles must be checked. Previously,
upstream bundles were not checked during bundle deletion which prevented
downstream mixes from detecting upstream bundle deletions.
Fixes#694
Signed-off-by: John Akre <john.w.akre@intel.com>
Currently, with the use of go modules, packaging using autospec expects
all the vendored dependencies to be available as go modules in koji.
Since that is not the case, the vendor directory is still maintained in
this repo.
By default, go commands like `go install` ignore the vendor directory
when in module mode. The -mod=vendor flag instructs the go commands to
use the main module's top-level vendor directory to satisfy dependencies.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Run `go mod vendor` to ensure that the vendor directory is updated.
NOTE: The vendor directory is still maintained because it is required
for packaging using autospec. It can be fully removed from this repo
only if all the vendored dependencies are made available as modules in
koji.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Now the mixer repo list command displays repo priorities and the mixer
repo add command can set the repo's priority with the --priority flag.
Signed-off-by: John Akre <john.w.akre@intel.com>
The set-priority repo command updates the priority value for the
specified repo in the DNF config file. This command uses the following
format:
mixer repo set-priority <repo> <priority>
Fixes#691
Signed-off-by: John Akre <john.w.akre@intel.com>
The functions to set values in the DNF config were nearly identical.
This change consolidates the repo set functionality into a single
function to increase code re-use.
Signed-off-by: John Akre <john.w.akre@intel.com>
This release includes using `go mod` to manage dependencies along with
MCA enhancements and bug fixes.
Changes:
- MCA: Refactor diff results analysis section to make it easier to handle
special case manifest files.
- MCA: Add support for bundle tracking file.
- MCA: Print warnings to stderr.
- swupd: Retain previously deleted file entries when the previous manifest's
file list is longer than the new manifest's list.
- vendor: Use `go mod` to manage dependencies.
- Makefile: Remove unused local gopath and set default gopath.
- Makefile: Install go linters if its not present.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
A local gopath is no longer needed to build if the code is outside the
go src/ folder. This change also allows `make lint` to work if the code
is outside the go tree.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
go 1.11 has added built in support to manage vendored dependencies.
Using go mod also allows mixer to be built if the source is outside
GOPATH.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
When the previous manifest's list of files was longer than the new
manifest's list of files there was a chance for previously deleted file
entries to be unexpectedly dropped. This change prevents previously
deleted files from dropping when the previous manifest's file list is
longer than the new manifest's list.
Fixes#684
Signed-off-by: John Akre <john.w.akre@intel.com>
The bundle tracking file is added to each manifest by mixer and does not
exist in an rpm. This change adds a check for the bundle tracking file
when adding a bundle.
Signed-off-by: John Akre <john.w.akre@intel.com>
The diff results analysis section was refactored to make it easier to
handle special case manifest files.
Signed-off-by: John Akre <john.w.akre@intel.com>
This release includes improved MCA output format along with improved
performance for repos with multiple sub directories.
Changes:
- vendor: Add tablewriter.
- MCA: Refactor statistics table.
- repo: Improve performance for multiple repo subdir.
- Fix lint errors.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
If an rpm is not found in the assumed cache dir of its repo, update the
repo cache dir list based on the location of the rpm.
This is an attempt to improve performance in case the rpms are not
stored directly within the repo baseurl.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Reagan Lopez