This release includes the following:
- Fix iterative manifests leaking on minversion
- Enable iterative manifests with 0 file count
- Add support for experimental bundles
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
From format 27 onwards, Mixer will recognize bundles with "[Status] : Experimental"
as experimental bundles and mark their status flag as "e" in the Manifest.MoM.
i.e. These bundles will include an "e" in the 2nd position of the manifest flags,
as shown here:
Me.. 4b91c3122e7e32f1e3edb597c6f89ff32cfffd977afb5e5 10 some-bundle
Fixes#497
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
This test verifies that iterative manifests aren't unintentionally
generated or forwarded into new MoMs by a minversion update.
Signed-off-by: John Akre <john.w.akre@intel.com>
During a minversion update, iterative manifests older than the new
minversion were leaked into the new MoM. Also, format bumps perform
minversion updates, so iterative manifests from an old format could
leak into new formats. This change prevents forwarding iterative
manifests with versions older than the minversion into the new MoM.
Fixes#523
Signed-off-by: John Akre <john.w.akre@intel.com>
When a bundle is updated to only modify which bundles are included, an
iterative manifest can be generated with 0 files. This change adds a
type field to the Manifest struct to determine when an iterative
manifest is used and allows the creation of iterative manifests with
a 0 file count.
Fixes#516
Signed-off-by: John Akre <john.w.akre@intel.com>
This reverts commit c6777502c8.
To make this automatically packageable through autospec we need to do
some more work. The travis environment also needs to be trimmed down to
speed up the build, and building the package should work without network
connection. In the current state, it will not satisfy both travis and
autospec, so reverting this until it can be properly transitioned.
This release includes the following fixes and additions:
- Fix MoM verification with OpenSSL 1.1 requring a purpose to be passed
- Make --config global
- Break root preRun() into smaller parts, with better & simplified logic
- builder: only check online to update upstreamver
- Removed editor option for bundle edit; simply creates new bundles or
copies existing ones now
- Added bundle_control unit tests
- Convert to using go mod vendor instead of our full vendor directory
- Enable iterative manifest creation for more optimal update content
creation
- Misc formatting and error fixes
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The format guard test for the tranisition from format 25 to 26 became
too large. This test has been split into separate tests for each
feature. These features include minversion, iterative manifests, delta
manifests.
Signed-off-by: John Akre <john.w.akre@intel.com>
Similarly to iterative manifests, delta manifests were introduced in
format 26 and should not be used in earlier formats. This change
restricts delta manifest generation in unsupported formats and updates
delta manifest tests to use supported formats.
Signed-off-by: John Akre <john.w.akre@intel.com>
Iterative manifests should only be used on manifests with formats
greater than 25. Adding a format guard to prevent iterative manifest
generation on formats less than or equal to 25. Additionally, this
change updates iterative manifest tests to use formats greater than 25
and adds a test to verify that older formats do not generate iterative
manifests.
Signed-off-by: John Akre <john.w.akre@intel.com>
Currently, when updating from version X to Y, in order to apply a delta, swupd
needs to download the Manifest from version X to check which file the
from-hash in delta file is pointing to. Downloading that Manifest some times
is even larger than the delta pack, so reducing the download size would make
a significant improvement in update time.
The Delta-Manifest is a Manifest included in delta packs with all information
from Manifest X in order to apply deltas, i.e., all files that are going to be
used to apply deltas.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When renaming, don't override file type in the from-manifest. This
information isn't used anywhere in mixer and it is necessary to create
delta manifests.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When we have a minversion, all files from all Manifests are considered
updated and have its version set to the minversion number. In this case
Iterative Manifests will include all files from the full bundle Manifest.
So there's no gain in creating iterative manifests in a minversion.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When creating manifests for a new build, also create to-Manifests
for each bundle that has changes in this version.
The goal on publishing to-Manifests is to reduce the download size
on updates, according to what was discussed on
clearlinux/swupd-client#498
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The clear repo is mandatory for mixin. This change prevents mixin from
removing the clear repo.
Fixes#326
Signed-off-by: John Akre <john.w.akre@intel.com>
`bundle add` will now add valid bundles and skip only the invalid ones,
thereby fixing issue #489.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
go mod vendor cleans up the vendor directory to only keep what source
files are actually needed.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
This means we can remove local gopath preparation from the Makefile and
will be using the officially sanctioned dependency management system for
Go. We have to leave the vendor directory so we can create the dist
tarball.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Some testcases showed these checks were incomplete and not handling
corner cases. Improve the logic of these checks to comply with the new
tests.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Instead of requiring upstream bundles be read from a child directory
under where mixer is currently run read them from underneath the mixer
workspace (builder.Config.Mixer.VersionPath) the same way local-bundles
is handled.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Start to add some unit tests for the bundle_control file. There is much
work to still be done as far as testing goes for these methods.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
- Reordered the tests to reduce code change impacts
- Added more test coverage for `bundle remove`
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
With the removal of the editor feature in `bundle edit`,
the term `edit` is not relevant anymore. Hence, changing it to `create`.
In order to prevent breakage of existing use-cases,
`edit` is aliased to `create`and `--suppress-editor` is made hidden and deprecated.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
`bundle edit` will no longer open the editor. It will just create new bundles or copy existing bundles.
This command will locate the bundle by first looking in local-bundles, and then in upstream-bundles.
If the bundle is only found upstream, the bundle file will be copied to the local-bundles directory.
If the bundle is not found anywhere, a blank template will be created with the correct name.
This commit also fixes issue #448 and increases test coverage for `bundle validate`.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Updating mixver should not require a network connection, nor reach out
to the server to find out any more information than is needed.
Fixes#282
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
With the exception of init, all other subcommands have their own file.
This patch moves init to a separate file to keep it consistent with the
rest of the code.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The root preRun() function accumulated a lot of responsibility over time
that are better located into sub command. This PR deconstructs that
function in order to have a simpler codepath.
Now, the prerun performs only 3 actions:
- Start profiling if needed.
- Check if it is the root command.
- Check if the command should run on a container or not.
The container logic has also been simplified. All commands are not
treated as native commands. During the pre-run, if a command has the
container marker, the current execution is aborted and the command is
re-executed inside a container. This way, the native flag only needs to
be checked once during execution.
Most of the logic in the root preRun() were exclusive to the `build`
subcommand, so they were moved to the build preRun().
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
The --config flag was added to every single subcommand in mixer. Instead
of doing so, make it a global flag.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
When mixer is executed without any argument, it should print its usage
message and exit. In order to do so, mixer must skip all it's checks on
the pre-run and just return if the current command is the top level
command alone.
Signed-off-by: Rodrigo Chiossi <rodrigo.chiossi@intel.com>
This release of mixer includes various changes around the entire much of
the code base:
- Mixer uses the log package for warnings and errors throughout the code
to more properly handle writing to correct streams.
- More BAT tests have been added to expand functional test coverage.
- Many unit tests have been added to start getting better coverage for
the builder package.
- Builder package was broken up from an almost 3000 line file into
multiple, logical files that are easier to work with.
- Mixer ignores validation for bundle list so it does not fail to list even
if there is an invalid bundle.
- Increased bsdiff delta timeout to handle large files which *do* create
deltas, despite the increase in build time. These are typically large
GCC files which cannot be stripped at this time.
- The build all command supports --increment now instead of auto
incrementing the mixversion.
- Format bump process was updated to use a new simplified flow, which
does not require switching tooling back and forth during the build. This
process is used to peform both upstream and downstream format bumps
automatically for the user.
- Build image automatically gets the ister template if it does not exist
and populates it with the bundles from mixbundles.
- No delta packs are created over format bumps. Minversions should have
delta packs created over them, but not format bumps because clients
can't update over bump boundaries.
- Misc bug and code structure fixes.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
There is no reason to number these tests as bat does not respect the
numbering system and it was obnoxious to try to keep collisions from
happening and keep the descriptions up-to-date.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
We had three 09-* tests, renumber them so we can prove we actually know
how to count when people look at our test directory.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Fixes#482
Since clients can't update over format bump boundaries anyways do not
attempt to create delta-packs covering that jump.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Add a bat test that creates a format bump then attempts to create delta
packs over that format bump. Make sure that the delta packs are not
created and that they are not even attempted to be created.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
This commit fulfills enhancement #58. When building an image without providing a template,
if the default image template is not present in the mix directory, mixer will do the following:
- Copy image template from ister template "/usr/share/defaults/ister/release-image-config.json".
- Update image template with the bundle list from "mixbundles" file.
In case the "mixbundles" file is empty or not present, use ister template as is.
- Inform the user about the above steps.
- Continue the existing build process.
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
Users basing their mixes on upstream Clear and not acting as pure OSV's
need to have the correct mixes automatically generated for them. This
creates the two mixes as needed and then resets their upstreamversion to
the version they were attempting to build.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Rodrigo Chiossi