This release fixes a subtle bug in try_delta_manifest_download() that would
incorrectly set the file struct causing hashes to become incorrect. It further
fixes problems in the retry path, in which it would loop continuously until
it was able to load a correct manifest.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The update code did not jump to the correct points when it needed to retry,
causing swupd to run into infinite loops if it could not load a given manifest
at all.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The new file struct cannot be populated using the original because it will
contain attributes and data that may not match the new file, breaking hash
computation. We must freshly populate the struct with the new file only
after bsdiff application succeeds, and then compute hash to ensure all data
is indeed from the new file. It is also irrelevant to precompute the hash
before attempting to apply the delta, so we remove that code block
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release fixes incorrect download messages printing for packs, validation
for the format number supplied to the binary, and enables delta file
application for manifests. Manifest deltas can be more than 150 times smaller
than the full file tar, saving time on updates by downloading much less data.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The tarfiles should not be kept after swupd runs, so check for the fullfile
instead and skip having to extract it.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
If there is no peer or we cannot get a delta, quit and don't mess with the
file struct. Pre-populating the file struct then exiting early corrupts the
manifest list for later operations, and is a noop when the file cannot be
found to begin with. If there is no peer, it likely is a new manifest and
thus a delta cannot exist.
Should the delta application succeed, we MUST compute and set the new hash
for the file else it will retain the hash of the previous version, and not
pass hash check.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current implementation is not setup to optimally load manifests and attempt deltas.
This changes the flow to be as follows:
1.) Load current Manifest.MoM
2.) Load server Manifest.MoM
3.) Recurse and load all current manifests, since we know exactly which are needed already
4.) Link current and server Manifest.MoMs, setting versions and peers
5.) Load server manifests recursively adding their includes
6.) Recurse the server manifests to find and load any new included bundles
7.) Link current and server Manifest.MoMs again to account for new bundles
When loading the server manifests, deltas will attempt to be applied to
current manifests before doing a full download. Following this flow enables
swupd to load manifests in a logical order, and short circuit trying to
reload them later on as manifests are recursed for includes again later.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current version must be passed to add_subscriptions, else it will call
load_manifest() with equal current and server versions and never enter the
try_delta_manifest code path.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
To ensure validity of the format number hardcoded in the binary, check
the supplied input against a regular expression using grep.
Note that the double square brackets are needed for the regex character
classes because square brackets are used for M4 quotation in Autoconf;
an extra pair of square brackets is needed to prevent them from being
stripped in the configure script.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Swupd incorrectly says "Downloading pack <name>..." even if it does not
really download the pack. Fix this by saying downloading packs and only
printing if we downloaded a pack to extract.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release adds proper systemd-reexec and daemon reload calls which must
be called without --no-block, such that triggers called after update are
the new triggers.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
We must not run these with --no-block because order must be preserved,
and critical components may need to be reexec'd, making this a core piece
of the swupd update process.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release introduces some large changes,
Features:
- swupd -t/--time flag to enable verbose timing output of swupd operations
(update, verify, bundle-add)
- verifytime service added to boot to check client systems are on a sane time
- Accept multiple bundles when using bundle-remove
Fixes:
- Attempt to automatically fix statedir corruption from bad files/tars
- Fix download and hysteresis logic on error paths and improve curl usage
- Improve return codes for bundle-add to cover various scenarios of
incorrect/correct bundle additions
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release introduces some large changes,
Features:
- swupd -t/--time flag to enable verbose timing output of swupd operations
(update, verify, bundle-add)
- verifytime service added to boot to check client systems are on a sane time
- Accept multiple bundles when using bundle-remove
Fixes:
- Attempt to automatically fix statedir corruption from bad files/tars
- Fix download and hysteresis logic on error paths and improve curl usage
- Improve return codes for bundle-add to cover various scenarios of
incorrect/correct bundle additions
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This is a wrapper around remove_bundle()
to add consistency for bundle-remove
subcommand respect bundle-add that accepts
one or more bundles to be removed.
This must not be the optimal implementation
but it works fine and gives a better user
usage experience.
To give libcurl an extra opportunity to do more work, add an additional
curl_multi_perform() call interleaved between curl_multi_wait() and the
attempt to process completed transfers.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The "numfds" variable set by curl_multi_wait() does not appear to be a
reliable indicator of how many transfers to try popping of the stack. I
went with this approach because the documented BKM, namely to check if
the "running" variable changes between calls to curl_multi_perform(), is
a bit inconvenient to integrate with swupd's code flow here.
Instead, try to empty the multi stack down the hysteresis lower bound,
and only check "numfds" to detect no activity at all. This approach
seems to perform better, resulting in a more balanced sequence of
additions and removals from the stack over time.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If a system loses network connectivity in the middle of fullfile
downloads for update (or verify --fix), and there are more pending
transfers to queue than remaining available file descriptors for the
process, the hysteresis upper bound is bypassed and swupd runs out of
open file descriptors.
To avoid this issue, the upper bound (plus a small buffer for leeway)
should be enforced in poll_fewer_than(). Since there is little
visibility into the state of libcurl as to why no downloads are
progressing, if the upper bound is bypassed, simply append the most
recently added file to the failed list.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
There may be in-progress transfers in the multi queue if
poll_fewer_than() returns an error, so a full clean of the multi queue
should be deferred.
The queue is already cleaned up immediately before the next download
retry loop begins; this is the best place for cleanup, since all
transfers have either completed or failed before the cleanup is
triggered.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If one of the poll_fewer_than() calls returned an error, a jump to the
out_bad label did not occur. Fix this issue, and also check error codes
the same way for both call sites.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Because the pointer in the struct file may have been NULL'ed previously
in free_curl_list_data(), the "curl" pointer declared on the stack is
not updated.
Since the free_curl_list_data() function already checks struct file
pointer instead, call it instead. swupd_download_file_complete() is also
called by free_curl_list_data(), so that call has been removed here.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Under certain error conditions for download retries, the curl multi list
cleanup routines are called more than once, leading to double frees for
CURL object pointers.
To protect against this problem, ensure that the pointers are reset to
NULL following the frees.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
For various reasons - partial downloads, power failure, user exiting -
the /var/lib/swupd state directory may become corrupt. When this happens,
manifests can be rendered unusable, causing various errors in swupd such as
signature failures, or errors reading the manifest. This patch attempts to
avoid such things causing updates or verifies to fail by cleaning up the
Manifests artifacts of what it could not use, and re-downloading them to
try again.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Arrange for "bad names" to give non-zero exit code
Arrange for "do nothing" in particular when packages are already
installed to give a zero return code.
This means that typos in names result in failures, but that trying to
add an existing package is fine.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
The verbose_time flag should be set to false if getting time fails, so timing
is not attempted again later, letting swupd continue in its regular mode.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This will ensure the system has a more correct, usable time after boot,
which not only improves swupd reliability but other services' as well.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
There may be code doing something like:
grabtime_start(×, "Outer loop");
. <do work>
grabtime_start(×, "Inner loop");
.
. <do work>
.
grabtime_stop(×);
. <do work>
grabtime_stop(×);
The "Outer loop" section would not have been printed because the stop time
and completed flag would not have been set. This fixes it by checking if we
already completed the last time grab, and traverses backwards to find the
next incomplete one, which must be the outer one.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This introduces a -t/--time option to update, verify, and bundle-add commands, displaying raw elapsed time and CPU process time for integral swupd operations within the given code paths.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release fixes a mismatch in the help menu description and long option,
and fixes the bash backwards compatibility check.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release updates and converts the man pages to RST format, updates the
swupd bash generation script, fixes error reporting when invalid bundle names
are given to bundle-add, and introduces a new tool to verify system time and
attempt to fix it if needed so signature verification does not fail due to
system time errors.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The system clock may be terribly off, especially on new hardware that has not
yet been calibrated. Updates rely on the certificate and system time being
sane to verify validity, so if a mismatch is found the certificate will
be deemed invalid and the update stopped. This patch attempts to fix the
system time to something sane using the time from the swupd binary itself,
which should not have been touched by any user except root. If the time is
normal and verification fails, the cert cannot be trusted.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Currently if you try and add a non existent bundle, e.g. "foo" you get
two lines of output
foo bundle name is invalid, skipping it...
bundle(s) already installed, exiting now
This is caused by the add_subscriptions function calling itself
recursivly but failing to pass up results in a meaningful way. This
change makes the return value of add_subscriptions be a bitmask so it
can signal errors and packages added distinctly.
I did think about changing this function to return a struct but
decided this was a step too far.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
I noticed this printf says something about getwd(), but that's not
actually being called directly there and that's not super useful to a
non-developer human anyway. So I translated the function name to an
english description.
Signed-off-by: Tim Pepper <timothy.c.pepper@linux.intel.com>
bash 4.4 added the 'nosort' option for completion. Use this so the
flag options come before the bundle names.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
Include feature request from IRC to not offer os-core and
os-core-update bundles as completion targets for bundle-remove.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
Remove the script which creates the completion script based on the
output of swupd --help. It wasn't being used (e.g. the completion
didn't have bundle-list in it).
Restructure the completion script to use a case statement to list the
valid completion options. IMHO this makes the code easier to
understand.
Add in package name completion for bundle-add. This requires
/var/lib/swupd/XXXXX/Manifest.MoM to exist (where XXXXX is the
contents of /var/lib/swupd/version), be in the correct format
etc.
Add in package name completion for bundle-remove. This uses the
contents of /usr/share/clear/bundles to get the list of installed
bundles. It would be nice to use $(swupd bundle-list) but it aborts if
it is not being run as root, so this means you can't have completion
for "sudo swupd bundle-remove".
TODO: Fix bundle completion if --path is specified.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
This release removes unused certificates from the client, and fixes the
makefile incorrectness that occured in the previous release.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The client does not do certificate pinning anymore, and it should not provide
certificates. This patch removes the unused certificates and clarifies the
certpath option.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Tudor Marcu