This release introduces the verify --picky subcommand, which lists files under
/usr, a new subcommand "autoupdate" which allows for easy enable/disable
of autoupdates and its current status, and converts printfs to fprintf to
stderr throughout the code as appropriate.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Add a "swupd verify --picky" command which lists files under
/usr (modified by the --path option) which are not listed in the
current manifest. Exceptions /usr/local and /usr/lib/modules. The
former to allow for local changes, the latter because the clear boot
manager owns that directory.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
Write error and informative messages to stderr, only
write data to stdout so it can be piped to other programs.
Make stdout line buffered, rather than introduce race conditions into tests
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
This release fixes a bug in signature verification retries, and adds support
to client so it understands a new manifest header field called "action." The
new field will allow client to detect if more actions need to be taken after
an update, i.e run verify --fix.
Signe-off-by: Tudor Marcu <tudor.marcu@intel.com>
The update process should only read in post update action from the newest
MoM. This makes it so that even with a previous MoM having an action, only
the new one will be taken, and if none exist, the old one will be ignored
so swupd will not take the same action as the previous update.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The swupd client should support parsing extra data in the case that a format
bump occurred, or other possibly breaking change, so that it can at least
notify the user of more action needing to be taken post update.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Unlike the regular manifest loading code, loading a MoM also requires signature
verification to succeed. We cannot reset the retries after a signature
failed because the manifest may load fine, but still fail verification,
resulting in endlessly loading and verifying the Manifest.MoM
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release adds retries to pack downloads for bundle-add. The downloads
could fail if network connectivity is lost, which drops the code into a
slower download path. Instead, attempt to retry pack downloads to stay on
the optimal code path before finally dropping into the fullfile fallback.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The download_subscribed_packs() could fail for network issues or other related
problems, and because the only option for getting new packs is by downloading
zero packs or fullfiles, swupd should retry to get the packs before falling
into the verify_fix_path flow which signifies pack downloads errored out.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release fixes a subtle bug in try_delta_manifest_download() that would
incorrectly set the file struct causing hashes to become incorrect. It further
fixes problems in the retry path, in which it would loop continuously until
it was able to load a correct manifest.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The update code did not jump to the correct points when it needed to retry,
causing swupd to run into infinite loops if it could not load a given manifest
at all.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The new file struct cannot be populated using the original because it will
contain attributes and data that may not match the new file, breaking hash
computation. We must freshly populate the struct with the new file only
after bsdiff application succeeds, and then compute hash to ensure all data
is indeed from the new file. It is also irrelevant to precompute the hash
before attempting to apply the delta, so we remove that code block
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release fixes incorrect download messages printing for packs, validation
for the format number supplied to the binary, and enables delta file
application for manifests. Manifest deltas can be more than 150 times smaller
than the full file tar, saving time on updates by downloading much less data.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The tarfiles should not be kept after swupd runs, so check for the fullfile
instead and skip having to extract it.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
If there is no peer or we cannot get a delta, quit and don't mess with the
file struct. Pre-populating the file struct then exiting early corrupts the
manifest list for later operations, and is a noop when the file cannot be
found to begin with. If there is no peer, it likely is a new manifest and
thus a delta cannot exist.
Should the delta application succeed, we MUST compute and set the new hash
for the file else it will retain the hash of the previous version, and not
pass hash check.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current implementation is not setup to optimally load manifests and attempt deltas.
This changes the flow to be as follows:
1.) Load current Manifest.MoM
2.) Load server Manifest.MoM
3.) Recurse and load all current manifests, since we know exactly which are needed already
4.) Link current and server Manifest.MoMs, setting versions and peers
5.) Load server manifests recursively adding their includes
6.) Recurse the server manifests to find and load any new included bundles
7.) Link current and server Manifest.MoMs again to account for new bundles
When loading the server manifests, deltas will attempt to be applied to
current manifests before doing a full download. Following this flow enables
swupd to load manifests in a logical order, and short circuit trying to
reload them later on as manifests are recursed for includes again later.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current version must be passed to add_subscriptions, else it will call
load_manifest() with equal current and server versions and never enter the
try_delta_manifest code path.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
To ensure validity of the format number hardcoded in the binary, check
the supplied input against a regular expression using grep.
Note that the double square brackets are needed for the regex character
classes because square brackets are used for M4 quotation in Autoconf;
an extra pair of square brackets is needed to prevent them from being
stripped in the configure script.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Swupd incorrectly says "Downloading pack <name>..." even if it does not
really download the pack. Fix this by saying downloading packs and only
printing if we downloaded a pack to extract.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release adds proper systemd-reexec and daemon reload calls which must
be called without --no-block, such that triggers called after update are
the new triggers.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
We must not run these with --no-block because order must be preserved,
and critical components may need to be reexec'd, making this a core piece
of the swupd update process.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release introduces some large changes,
Features:
- swupd -t/--time flag to enable verbose timing output of swupd operations
(update, verify, bundle-add)
- verifytime service added to boot to check client systems are on a sane time
- Accept multiple bundles when using bundle-remove
Fixes:
- Attempt to automatically fix statedir corruption from bad files/tars
- Fix download and hysteresis logic on error paths and improve curl usage
- Improve return codes for bundle-add to cover various scenarios of
incorrect/correct bundle additions
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release introduces some large changes,
Features:
- swupd -t/--time flag to enable verbose timing output of swupd operations
(update, verify, bundle-add)
- verifytime service added to boot to check client systems are on a sane time
- Accept multiple bundles when using bundle-remove
Fixes:
- Attempt to automatically fix statedir corruption from bad files/tars
- Fix download and hysteresis logic on error paths and improve curl usage
- Improve return codes for bundle-add to cover various scenarios of
incorrect/correct bundle additions
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This is a wrapper around remove_bundle()
to add consistency for bundle-remove
subcommand respect bundle-add that accepts
one or more bundles to be removed.
This must not be the optimal implementation
but it works fine and gives a better user
usage experience.
To give libcurl an extra opportunity to do more work, add an additional
curl_multi_perform() call interleaved between curl_multi_wait() and the
attempt to process completed transfers.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The "numfds" variable set by curl_multi_wait() does not appear to be a
reliable indicator of how many transfers to try popping of the stack. I
went with this approach because the documented BKM, namely to check if
the "running" variable changes between calls to curl_multi_perform(), is
a bit inconvenient to integrate with swupd's code flow here.
Instead, try to empty the multi stack down the hysteresis lower bound,
and only check "numfds" to detect no activity at all. This approach
seems to perform better, resulting in a more balanced sequence of
additions and removals from the stack over time.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If a system loses network connectivity in the middle of fullfile
downloads for update (or verify --fix), and there are more pending
transfers to queue than remaining available file descriptors for the
process, the hysteresis upper bound is bypassed and swupd runs out of
open file descriptors.
To avoid this issue, the upper bound (plus a small buffer for leeway)
should be enforced in poll_fewer_than(). Since there is little
visibility into the state of libcurl as to why no downloads are
progressing, if the upper bound is bypassed, simply append the most
recently added file to the failed list.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
There may be in-progress transfers in the multi queue if
poll_fewer_than() returns an error, so a full clean of the multi queue
should be deferred.
The queue is already cleaned up immediately before the next download
retry loop begins; this is the best place for cleanup, since all
transfers have either completed or failed before the cleanup is
triggered.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
If one of the poll_fewer_than() calls returned an error, a jump to the
out_bad label did not occur. Fix this issue, and also check error codes
the same way for both call sites.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Because the pointer in the struct file may have been NULL'ed previously
in free_curl_list_data(), the "curl" pointer declared on the stack is
not updated.
Since the free_curl_list_data() function already checks struct file
pointer instead, call it instead. swupd_download_file_complete() is also
called by free_curl_list_data(), so that call has been removed here.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Under certain error conditions for download retries, the curl multi list
cleanup routines are called more than once, leading to double frees for
CURL object pointers.
To protect against this problem, ensure that the pointers are reset to
NULL following the frees.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
For various reasons - partial downloads, power failure, user exiting -
the /var/lib/swupd state directory may become corrupt. When this happens,
manifests can be rendered unusable, causing various errors in swupd such as
signature failures, or errors reading the manifest. This patch attempts to
avoid such things causing updates or verifies to fail by cleaning up the
Manifests artifacts of what it could not use, and re-downloading them to
try again.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Arrange for "bad names" to give non-zero exit code
Arrange for "do nothing" in particular when packages are already
installed to give a zero return code.
This means that typos in names result in failures, but that trying to
add an existing package is fine.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
The verbose_time flag should be set to false if getting time fails, so timing
is not attempted again later, letting swupd continue in its regular mode.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This will ensure the system has a more correct, usable time after boot,
which not only improves swupd reliability but other services' as well.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
There may be code doing something like:
grabtime_start(×, "Outer loop");
. <do work>
grabtime_start(×, "Inner loop");
.
. <do work>
.
grabtime_stop(×);
. <do work>
grabtime_stop(×);
The "Outer loop" section would not have been printed because the stop time
and completed flag would not have been set. This fixes it by checking if we
already completed the last time grab, and traverses backwards to find the
next incomplete one, which must be the outer one.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This introduces a -t/--time option to update, verify, and bundle-add commands, displaying raw elapsed time and CPU process time for integral swupd operations within the given code paths.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release fixes a mismatch in the help menu description and long option,
and fixes the bash backwards compatibility check.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release updates and converts the man pages to RST format, updates the
swupd bash generation script, fixes error reporting when invalid bundle names
are given to bundle-add, and introduces a new tool to verify system time and
attempt to fix it if needed so signature verification does not fail due to
system time errors.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Tudor Marcu