This release includes bug fixes to maintain feature parity with v5.9.1
along with the MCA change to take into account sha256 hashes.
Changes:
- Skip hash comparisons between md5 and sha256 during `build validate`
to avoid incorrect hash mismatches. rpm 4.12 used md5 file hashes
by default and rpm 4.14 switched the default to sha256.
- Stop building bundles and exit if package dependencies are not resolved.
This re-enables feature parity with v5.9.1.
- Allow users to set multiple repos correctly by removing the
constraint to use only "clear" and "local" repos. This re-enables
feature parity with v5.9.1.
- Fix the incorrect error message when trying to remove a non-existent
repo.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Currently mixer allows users to set multiple repos using 'mixer repo set' commands and name them anyhow.
But the code within the mixer is tightly coupled to having only two repos - "clear" and "local".
This inconsistency is fixed.
fixes#654, #659
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
Currently, `mixer repo remove` displays incorrect messages if repo does
not exist.
This fix displays the correct message.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Currently, if dependencies are not resolved for a bundle, mixer ignores
it and continues processing other bundles. The user is not notified and
results in undesired output.
This fix ensures that the processing is stopped if dependencies are not
resolved and generates an error message.
Empty bundles are considered valid and does not stop processing.
Fixes#663
Signed-off-by: Ashlesha Atrey <ashlesha.atrey@intel.com>
rpm 4.12 used md5 file hashes by default and rpm 4.14 switched the
default to sha256. As a result, in version 31680 of Clear, rpms started
to use sha256 hashes instead of md5. Since MCA assumes that hash changes
indicate content changes, hash comparisons between md5 and sha256 are
skipped to avoid false positive errors.
Signed-off-by: John Akre <john.w.akre@intel.com>
This release includes a minor bug fix in the zero pack creation.
Changes:
- Remove shared variable in the zero pack go routine.
The err variable was being shared and updated in the zero pack go routine.
This could cause race conditions and undesirable results including segmentation fault.
This fix creates a local variable within the zero pack go routine.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
This release includes bug fixes and improved logging along with
replacing ister with clr-installer to build images.
Changes:
- Replace ister with clr-installer to build images.
Also generates an iso image.
- Preserve config filename during conversion.
- Use "dnf repoquery" to determine rpm filename instead of assuming the
naming convention.
- Identify missing rpms early on instead of waiting till extraction.
- Fix array out of bounds exception while downloading rpms.
- Fix incorrect logging during rpm extraction.
- Improve logging.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Switch from ister.py to clr-installer for generating OS images.
Will migrate existing ister JSON file if it exist.
Both ISO generate and Raw Image retention enabled by default.
Signed-off-by: Mark D Horn <mark.d.horn@intel.com>
If rpm is not found in the cache dir of "clear-*",
fail early instead of failing during extraction.
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
This fix avoids assuming the filename of the rpms in the "local" repo.
It uses "dnf repoquery" to fetch the rpm filename.
Fixes#648
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
When loading default values during conversion, the filename property was
also overwritten. Since the file should be converted in-place, the
original filename must be preserved.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This release includes major performance improvements to mixer build
process and removes some of its legacy systems.
On the performance side, RPM extraction is now performed by rpm2archive
and tar instead of dnf. This cuts a significant amount of time from the
overall build. Selective compression for fullfiles have also been
introduced and the default compression method is set to xz only. This
change reduces fullfiles generation time to 30% of the previous time at
the cost of 0.3% increase in build size for a mix with all upstream
packages. Lastly, zero packs creation have been parallelized to further
reduce build time.
Iterative manifests are now gone. Our tests showed that they didn't
present any performance advantage in comparison with delta manifests
which are already being used in production. The 'bundle index' bundle
has also been removed. This feature was never used for its intended
purpose and caused builds to fail frequently due to hash mismatches
during its generation. Existing mixes with this bundle will just have it
propagated as a empty bundle until the next format bump. At that point,
the bundle will be automatically removed.
Built in support for Docker has also been removed. The `--native` flag
has been deprecated and all commands now run in a native setup. Docker
support was introduced to allow auto-format-bump feature to be
implemented, but current mixer implementation does not require it
anymore to work. External docker images, like mixer-ci, should be used
as a replacement in use cases where the built-in docker support was used
to create mix in incompatible host OSes.
Mixer now provides the possibility of customizing the URL where the
bundles are downloaded from. Previously that was hardcoded to upstream
bundles, which prevented mixes from being created based of downstream
releases. This option can be configured in 'builder.conf'.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Improve performance of zero pack creation by generating them in
parallel. Fixes#629
Although zero pack is technically a delta pack, the parallel code for
delta pack creation has not been reused because of the following:
- For zero packs, processing is stopped immediately in case of error.
Whereas for delta packs, processing is continued even in case of error
because delta packs are optional.
- The report and print statements are different for zero packs and delta
packs.
The delta-workers flag is used as the numWorker for zero packs.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Built-in Docker support was originally added to enable auto-format bump.
At that time, mixer could only handle one format version at a time, so
we needed 2 different binaries to cross a format boundary: 1 for the +10
and one for the +20. The docker support would allow us to download an
image with the right version for that given format and simplify the
process.
Mixer now has support for multiple formats and a single binary can
handle the whole format bump process. Built in docker image is no longer
needed.
Note: For Docker usage due to compatibility issues with the host system,
mixer-ci Docker image should be used instead.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Format mismatch between the host and upstream was an issue when a given
mixer version could only generate a single format. Since we introduced
multi-format capability, the format of the host doesn't influence the
build anymore and this check can be safely ignored.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
In getUpstreamBundles(),
- Remove the prune param as it is always true. Fixes#624
- Remove the version param as it is available in the method.
In getUpstreamBundlesPath(),
- Remove the version param as it is available in the method.
Also, remove the need for downloadUpstreamBundles() method by including
its code in getUpstreamBundles().
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Currently, the bundle defintions are downloaded from the default
clr-bundles repo https://github.com/clearlinux/clr-bundles/archive/.
The user can now specify a different upstream bundles url using a mixer init flag or mixer
config parameter.
E.g. mixer init --upstream-bundles-url=https://github.com/reaganlo/clr-bundles/archive/
E.g. mixer config set Swupd.UPSTREAM_BUNDLES_URL https://github.com/reaganlo/clr-bundles/archive/
Since there is a tight coupling between the --upstream-version and bundles
repo, the user should ensure that the bundles repo has the same structure as the
clr-bundles repo and the tar.gz files should be named after an upstream
version. i.e. If the mix is based on an upstream-version 31300, the bundles repo
should have a file 31300.tar.gz
Fixes#614
Note: Setting this value using `mixer config set` will only update it in builder.conf.
It does not fetch the upstream bundles. A dedicated command for that purpose
will have to be implemented instead.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Generate an empty bundle os-core-update-index to replace the old index,
deleting all files included in previous manifests.
- if the previous index exists and it's not empty, we generate an empty index
- if the previous index doesn't exist we don't generate any index
- if the previous index is an empty index, we just point to it in the new MoM
Fix#541
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Tracking file and directory was being created with incorrect permissions.
This is important because tracking files have excatly the same hash, except
this one.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
For list values, if defaults are not loaded, they are assumed to be
missing and are not transferred to the converted file.
This patch changes the conversion behavior so that the default values
are always loaded before the config is parsed.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Add support to zstd for fullfile creation. This compression method can
be enable via config using option "external-zstd".
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This patch allows users to configure the compression methods that will
be used for fullfiles creation. The compression method can be set in the
Swupd.COMPRESSION property in builder.conf. Instead of enabling all
methods as previosly, xz is the only method that comes enabled by
default.
Note: internal gzip is still used for Link and Directory fullfile
creation.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Extracting rpm uisng rpm2archive is faster than dnf and will
improve performace significantly.
Fixes#631
Signed-off-by: Ashlesha Atrey ashlesha.atrey@intel.com
InitMix() calls AddBundles() and getUpstreamBundles().
AddBundles() already calls getUpstreamBundles().
So remove the redundant function call in InitMix().
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
When reading a MoM always ignore IManifests. This is important to prevent
IManifests to be carried out to next mom in an update
Also adding some deprecation warnings
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This commit was created based on reverting commits:
"Introducing the Delta-Manifests"
69fea81031.
"Add delta manifest format guard"
1c764e7765
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This patch expands mixer config type support to allow int and slice
values to be present in builder.conf. The expandEnv method is now
restricted to string variables and the `config set` command has been
expanded to allow comma separated lists as input for TOML lists.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This PR breaks bat tests into individual test runs within travis
infrastructure. This is a workaround the current test failures on travis
due to infrastructure limitations and allow us to reset a single test in
the case of a failure.
This patch also disables create-mix-bump-version-add-remove-bundles test
since it is unable to complete in travis infra.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
Update the man pages to reflect the latest content from the original content in
the .rst (ReST) documents.
Signed-off-by: Geoffroy Van Cutsem <geoffroy.vancutsem@intel.com>
Reagan Lopez