This change has proven to cause unnecessary churn for the bundle
tracking files when their contents ended up not being consumed by any
tools.
This reverts commit 6f840d712e.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Add the packages found in a bundle to the bundle tracking file. This
will be handy when trying to figure out client side what packages are
installed on the system.
Signed-off-by: William Douglas <william.douglas@intel.com>
This release includes minor bug fixes and enhancements.
- Fix generated warning for delta packs.
- Show upstream-format command in help.
- Remove delta file in case bspatch test fails.
- Update warning message for missing state file.
- Update to GitHub Actions v2.1.0
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
To avoid misleading info, indicate the correct warning message for
missing state file. Write the details of the default state values as
debug messages.
Fixes#491, #532
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
While creating delta packs, only possible delta files are found.
If delta file is not found, log the error in debug mode instead
of generating warning, as delta might not have been
created for valid reason.
fixes#628
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
This release includes minor bug fixes along with implementation of
logging.
Enhancements:
- Implement the logging package for mixer.
A common log file can be set for all the mixer commands in the
builder.conf. E.g. `mixer config set Mixer.LOG <filepath>`.
The log file and level can also be set for individual mixer
commands using the `--log` and `--log-level` flags respectively.
The various log levels are:
ERROR (1), WARNING (2), INFO (3), DEBUG (4) and VERBOSE (5).
Default log level is 4.
Other Changes:
- Remove the --no-publish flag.
Mixer will now always update the LAST_VER file after `build update`.
- Update github actions to v2 to handle CI re-runs.
- Update Makefile to build man pages as a prerequisite for the
`build` target.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Reduce cluttering the logs by logging command tags only for the external
applications used within Mixer.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Implement the logging package for mixer.
A common log file can be set for all the mixer commands in the
builder.conf. E.g. `mixer config set Mixer.LOG <filepath>`
The log file and level can also be set for individual mixer
commands using the `--log` and `--log-level` flags respectively.
The various log levels are:
ERROR (1), WARNING (2), INFO (3), DEBUG (4) and VERBOSE (5).
Default log level is 4.
Fixes#666
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
Update man page to include `format` flag.
Also, update Makefile to build man pages as a prerequisite for the
`build` target.
Fixes#557
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Mixer will now always update the LAST_VER file after `build update`.
This file is used by `build validate` (MCA) to determine whether
the version in comparison is a +10.
Fixes#735
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
This release includes performance improvements, minor bug fixes and
removal of the Mixin feature.
Changes:
- Remove the Mixin feature as Mixer now supports creation of 3rd party content.
- Calculate the hash for each file just once.
- Parallelize bundle processing.
- Parallelize bundle file resolving.
- Update mix variables when crossing upstream formats.
- Fix MCA bundle deletion during format bump.
- Add more debug messages for DNF errors.
- Fix permission issue in bat test.
- Replace Travis CI with GitHub Actions.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
In addition to printing a summary of unresolvable package names whenever
DNF fails, also print error messages for any "unknown" DNF error, and
finally print the full stdout/stderr from failed DNF commands for later
inspection.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
When bundle is deleted in a mix, bundle info file is not generated for that
bundle. When parsing the manifest, if bundle info is empty, consider that
bundle to be deleted in the mix and do not resolve packages for that
bundle during the MCA process.
Fixes#743
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
By creating a separate installroot for each package resolving goroutine,
dnf can run in parallel without contention on the lock file when
accessing the dnf cache. Now, package and file resolution occurs within
the same installroot so that the dnf cache that was fetched when resolving
packages can be re-used when resolving files.
Fixes#730
Signed-off-by: John Akre <john.w.akre@intel.com>
When a downstream mix crossed multiple upstream format bumps, the
upstreamversion and mix state variables were updated in the workspace,
but the changes were not updated in memory. This change reloads the
upstreamversion when it is staged/unstaged and reloads the mix state
after crossing each format boundary. Additionally, this change
increments the format number for each upstream format that the
downstream mix crosses.
Fixes#688
Signed-off-by: John Akre <john.w.akre@intel.com>
Improve the performance of processing bundles by parallelizing readIncludes()
and detectManifestChanges() operations.
Fixes#725
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
Currently, mixer build update performs multiple hash calculations for each file in the chroot.
Every manifest file entry and subtracted file will have their hashes calculated.
It would be a significant performance improvement to calculate the hash one time for each file and re-use it.
fixes#585
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
This release includes minor enhancements and bug fixes.
Enhancements:
- Initialize local yum repo even when its empty as newer versions of
DNF requires that yum repos be initialized.
- Install special case files in the end so that they will not create
directories with potentially conflicting permissions.
The order of installation is now packages, contents and then special
case files.
Bug Fixes:
- Set file permissions in the chroot after creating it and setting the
file owner. This is necessary because umask can prevent newly created
files from acquiring the source file's permissions.
- Don't follow symlinks when comparing files between content and full
chroot.
- Skip format bump check in MCA.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
When adding special case files to the full chroot, they will create
missing directories in their path with permissions that may conflict
with content that will be added at a later point. By adding special case
files last, they will not create directories with potentially
conflicting permissions.
Fixes#738
Signed-off-by: John Akre <john.w.akre@intel.com>
When comparing files between content and full chroots, do not follow
symlinks so that the links can be compared.
Fixes#739
Signed-off-by: John Akre <john.w.akre@intel.com>
When calling the copyFileWithFlags function with the preserveSrc flag
set, file permissions must be set after creating the destination file
and setting the file owner. This is necessary because umask can prevent
newly created files from acquiring the source file's permissions and
file ownership changes can impact the setuid/setgid permissions.
Similarly directory permissions are set after creating the directory to
avoid unexpected permission behavior caused by umask.
Signed-off-by: John Akre <john.w.akre@intel.com>
Newer versions of DNF require that yum repos are initialized, so Mixer
must initialize the local repo, even when it's empty.
Fixes#731
Signed-off-by: John Akre <john.w.akre@intel.com>
This release includes support for building 3rd party content along with
other enhancements and bug fixes.
New features (beta) for 3rd party support:
- Create a minimal mix using just an os-core bundle that can be empty.
- Add BUILD_ID field in os-release file that specifies the upstream
version from which the mix is created.
- Build bundles from contents by providing the content path using a new
keyword "content()" in the bundle definition.
- Introduce a new export flag "x" in bundle manifests to indicate exportable
files in "/bin/", "/usr/bin/" and "/usr/local/bin/".
The user can override this flag by specifying the files using a new
keyword "un-export()" in the bundle definition.
Other changes:
- Fix error to handle bundles that contain only optional bundles.
- Fix build errors due to stale dnf cache by cleaning the dnf cache before using it.
- Clean mix workspace before building bundles.
- Add --local flag for bundle creation that skips the upstream check and
creates empty local bundles.
- Improve performance by writing mixbundles file just once for the resolved bundle set.
- Make bundle messages consistent.
- Print delta pack results together to prevent interleaved results.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
When using content chroots, file uid/gid values should be preserved when
copying them to the full chroot.
Fixes#727
Signed-off-by: John Akre <john.w.akre@intel.com>
When resolving the full mix bundle set, the mixbundles file is now
updated once with the finalized set of bundles. Previously, the
mixbundles file was written many times with intermediate results which
scaled poorly when building with many bundles.
Fixes#724
Signed-off-by: John Akre <john.w.akre@intel.com>
When building a mix with a version of a package repo that contains
existing cache contents, dnf may resolve incorrect results.
For instance, Mixer does not version the local package repo, so when
incrementally building a mix with local content, dnf will continue to
re-use the same cache for the local repo. This was problematic when the
version of an rpm in the local repo was updated. In this case, the stale
cache metadata was used to incorrectly resolve files for the package.
Cleaning the dnf cache before using it prevents old cache metadata from
impacting dnf results.
Fixes#719
Signed-off-by: John Akre <john.w.akre@intel.com>
By default, files with the prefix "/bin/", "/usr/bin/" and "/usr/local/bin/"
will have a new export flag "x" in the 4th byte of the flag section of the
bundle Manifest.
The user can override this by using a new keyword "un-export()" in
the bundle definition file.
E.g. un-export(/usr/bin/file2)
Note: The export flag is at a bundle-file level and hence does not get
written to the Full manifest.
Fixes#708
Includes bat tests for the export flag feature.
Also, adds an entry for the content-chroot tests to Travis.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Change RenameFlag to MiscFlag since "rename" has been deprecated and
this flag will be used as a placeholder for various other flags.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Currently, build bundles generates an error if one of the mix bundles
has only an 'also-add()' entry in its definition.
Instead, build bundles should not error out and must process such
bundles.
Fixes#718
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Delta packs are generated concurrently, so their results output must be
printed once to prevent interleaved results. Additionally, this change
redirects warnings to stderr so that can be separated from the results.
Fixes#551
Signed-off-by: John Akre <john.w.akre@intel.com>
Passing '--local' with mixer bundle create will skip
the upstream check and create a new empty local bundle
if it does not already exist.
fixes#713
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
William Douglas