Use libarchive in order to make use of its security features and avoid
calling out to tar via a shell. The TAR_COMMAND is still used in
staging.c to complete the copy when a hardlink fails.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
The code implementing the suppression of /usr/lib/kernel,
/usr/lib/modules and /usr/local did a rather simplistic prefix
comparison with strncmp() to match path names against these
exceptions. As a result, paths like /usr/lib/kernel.old which should
get reported also got skipped.
An if check tested the wrong variable, so the result from strdup()
wasn't checked.
The replacement code uses regular expression matching and allows
replacing the default (--picky-whitelist
"/ignore-this-top-level-dir"). A single parameter is enough when using
the | operator.
A/B partition update must remove extra files in the entire partition,
which can be selected with --picky-tree=/, and also needs to override
the default whitelist.
Fixes#336
Implements #239
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
When content mismatches in the user manifest and the upstream content,
that is - the filenames are equal but the hashes are not, this signifies
that something is being introduced that already exists and is not the
same. Currently, only additive functionality is fully guaranteed to work,
so existing packages can be added as long as they are the same as the ones
provided by upstream. If non-unique content is added, all conflicts will be
printed out to the user so it can be fixed, and swupd will exit unless it
is given the allow-mix-collisions override flag.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Use defines instead, and check the user passed in URL against the URLs listed
in the official upstream contenturl file vs a static hardcoded URL.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This patch adds the logic required to handle files and manifests
differently if it is detected that the system is on a mix, or the
content being provided is local mix content.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Until internal data structures are fully migrated, provide a helper function
to transform a loaded manifest to an array. We can then do file lookups in
O(logn) time using binary search instead of traversing linked lists.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The filecount field becomes relevant when using an array implementation
of the manifest contents, so store it in the struct.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Add the --no-scripts flag to update, verify, and bundle-add commands to
allow users to skip the post update scripts for these commands. This is
useful when running in environments when systemctl is not available such
as containers.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Instead of relying on swupd-server to put the post-update action in the
correct manifest and in the correct format, do the format change
detection on the client.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
The check_network() function definition was recently removed, but the
declaration remained. Remove the declaration as well.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Implement handling of ghosted files in manifests via the 'g' flag. These
files are treated the same as deleted files except that they are not
actually deleted during the update. This allows ghosted files to serve
as the basis for renames while not actually being deleted by swupd.
This change also covers the verify --fix and verify --fix --picky paths.
Ghosted files will be ignored in these cases.
An example of where this is useful is with kernel files, which are
deleted from /usr/lib/kernel/ by clr-boot-manager but should still be
marked as deleted to utilize the rename functionality.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
A colon-separated list of alternative CApath options can be passed to
swupd-client at the configuration time using --with-fallback-capaths
option.
In runtime, fallback CApath support is implemented as part of the
connectivity check. The implementation will try the default (built into
curl) and then will iterate through the list in the order they were
specified. It is done only once on the first call.
The code is reorganized to keep the connectivity check inside curl
wrapper:
* Deleted check_network implementation from version.c
* Removed have_network global (globals.c)
* Scoped swupd_curl_test_resume to curl.c
* Change use of check_network to swupd_curl_check_network
Also:
* Fixed an issue where SSL was only enabled if a URL was matching the
content URL
swupd used to enforce that version and content URL had to use the same
protocol because then it only needed to check once whether it was
downloading locally.
However, it's not that much more difficult to check each individual
URL, so now that's what swupd does now and thus the restriction is
gone.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Since clr-boot-manager does not work in a container add the
--no-boot-update option to allow installation into a container to skip
the boot partition management. This also calls apply_heuristics when
running bundle-add, with the result of running CBM when necessary for
bundle-adds and skipping CBM when the --no-boot-update flag is
specified.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
When running verify --fix with -m specifying a different version than
the current OS version, require --picky to enforce management of files
under /usr or require --force as an override. Adding --picky can solve
issues such as issue #238 where fixing backward caused glibc library
confusion.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Allow user to list all bundle dependencies of a bundle passed to
swupd bundle-list --deps BUNDLE
This lists all included bundles (included those recursively included)
for the BUNDLE. This is particularly useful when applications or users
need to determine just how large the bundle addition will be. For
example, a user may not be aware that by bundle-adding desktop-dev they
are recursively adding 12 other bundles. This command allows the user to
make this check before installation.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
The --has-dep=BUNDLE argument will display a tree representing all
installed bundles that recursively include BUNDLE.
One may pass the --all argument as well to list the dependency tree for
BUNDLE including all installable bundles available on the server.
**Example output without --all:**
Installed bundles that have os-installer as a dependency:
format:
# * is-required-by
# |-- is-required-by
# * is-also-required-by
# ...
* mixer
|-- os-clr-on-clr
**Example output with --all:**
Attempting to download version string to memory
All installable and installed bundles that have os-installer as a dependency:
format:
# * is-required-by
# |-- is-required-by
# * is-also-required-by
# ...
* clr-devops
* mixer
|-- os-clr-on-clr
|-- os-clr-on-clr-dev
|-- clr-devops
|-- os-clr-on-clr-dev
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
When using verify's -m option, it generally only makes sense to verify a
build within the same format, since the subsequent format (if any) might
not be compatible. Therefore, we should disallow verifying to a build
with a different format by default. This commit adds the proper
enforcement by making this condition a fatal error. The early exit can
be bypassed using the -x/--force option, which will print a warning
message instead.
For now, this compatibility check is targeting the use case of verifying
to a newer build, so I also report the latest supported build for the
current format. Verifying to an older build is not guaranteed to succeed
at present, and swupd-client does not yet understand the
version/formatN/first file. In other words, reporting to the user the
oldest build they can verify to will be a future improvement.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Add a basic progress indication for a couple potentially long-running
swupd tasks (download_pack and add_missing_files). The progress
indication is displayed as a percentage:
...85%
It overwrites its own line so the progress will update in place. It may
be interrupted by other swupd output:
...45%
Extracting os-core pack for version 10000
...50%
Extracting os-core-update pack for version 9000
...55%
The progress indication will then resume updating in place, on the same
line.
This progress is only displayed when outputting to a TTY, and will not
output anything when the output is being redirected. This is nice for
testing and logging, but means that these progress updates will not be
caught by processes that capture swupd output, such as ister.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Instead of allowing only one "actions:" field in the Manifest.MoM, allow
for several and read each into the post_udpate_actions list. This allows
us to add more actions down the road if desired.
Currently the only use for the "actions:" field is to indicate when a
re-update is required. This is handled by checking if the string
"update" is in the post_update_actions list.
Finally, remove the warning to the user to perform the post update
action themselves, as this will be handled by swupd itself. Functional
tests updated to reflect the missing output.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Adds a check to compare version numbers in <state_dir>/version and
<target_dir>/usr/lib/os-release when attempting to re-update after a
format bump.
os-release should always be equal to <state_dir>/version for normal
updates and potentially greater than <state_dir>/version for format
bumps.
In the event of a bad build where os-release does not get updated to the
new version, this check will prevent an infinite loop where swupd reads
the current version from os-release, sees it is out of date, attempts to
update again, but the file does not get updated.
This occasion is very unlikely but the guard is nice to have
re(guard)less.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
When a device is out of date by more than a format bump re-exec swupd
update until the device version matches the server version. This is
achieved by reading the "actions" field in the Manifest.MoM, which is
yet to be implemented in swupd-server. When "update" is specified in the
actions of the current manifest and the current update is successful,
swupd will re-exec itself with the same flags it was originally called
with.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
We draw a hard line with error codes, such that:
Error code 16 is reserved for failures resulting from the basic
network check. Any call to check_network() that fails will
return this error code. No other error path returns this
error code.
Additionally, check_network() is called in every normal code path.
All other, possibly network related issues, return a different
error code. If the basic network check succeeds, but e.g. pack
downloads fail, we return a new (23) error code so that we
can better establish the conditions through telemetry and
determine whether the error is on the client or the server,
which is highly likely with this new error code 23.
This is the basic implementation for a "factory reset" type of command.
It will take the functionality of verify --picky and act on the files found
as verify --fix does with files found mismatching in the manifest.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Add a "swupd verify --picky" command which lists files under
/usr (modified by the --path option) which are not listed in the
current manifest. Exceptions /usr/local and /usr/lib/modules. The
former to allow for local changes, the latter because the clear boot
manager owns that directory.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
The update process should only read in post update action from the newest
MoM. This makes it so that even with a previous MoM having an action, only
the new one will be taken, and if none exist, the old one will be ignored
so swupd will not take the same action as the previous update.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The swupd client should support parsing extra data in the case that a format
bump occurred, or other possibly breaking change, so that it can at least
notify the user of more action needing to be taken post update.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current implementation is not setup to optimally load manifests and attempt deltas.
This changes the flow to be as follows:
1.) Load current Manifest.MoM
2.) Load server Manifest.MoM
3.) Recurse and load all current manifests, since we know exactly which are needed already
4.) Link current and server Manifest.MoMs, setting versions and peers
5.) Load server manifests recursively adding their includes
6.) Recurse the server manifests to find and load any new included bundles
7.) Link current and server Manifest.MoMs again to account for new bundles
When loading the server manifests, deltas will attempt to be applied to
current manifests before doing a full download. Following this flow enables
swupd to load manifests in a logical order, and short circuit trying to
reload them later on as manifests are recursed for includes again later.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current version must be passed to add_subscriptions, else it will call
load_manifest() with equal current and server versions and never enter the
try_delta_manifest code path.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
We must not run these with --no-block because order must be preserved,
and critical components may need to be reexec'd, making this a core piece
of the swupd update process.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
For various reasons - partial downloads, power failure, user exiting -
the /var/lib/swupd state directory may become corrupt. When this happens,
manifests can be rendered unusable, causing various errors in swupd such as
signature failures, or errors reading the manifest. This patch attempts to
avoid such things causing updates or verifies to fail by cleaning up the
Manifests artifacts of what it could not use, and re-downloading them to
try again.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This introduces a -t/--time option to update, verify, and bundle-add commands, displaying raw elapsed time and CPU process time for integral swupd operations within the given code paths.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The system clock may be terribly off, especially on new hardware that has not
yet been calibrated. Updates rely on the certificate and system time being
sane to verify validity, so if a mismatch is found the certificate will
be deemed invalid and the update stopped. This patch attempts to fix the
system time to something sane using the time from the swupd binary itself,
which should not have been touched by any user except root. If the time is
normal and verification fails, the cert cannot be trusted.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This sets swupd signature verification to mandatory, failing and exiting
for any verify error, though --nosigcheck enables user to bypass all
signature-based security checks.
The signed MoM serves as the top level chain of trust, and it is used to
extend content trust down to the individual file level. When the signature
of the top-level MoM is invalid or cannot be verified for any reason, we
warn and abort the operation.
Passing --nosigcheck will allow this to proceed, explicitly accepting the
unverifiable MoM and outputting a log entry to the Journal. This is not
recommended and unsupported by upstream once the chain of security is broken,
because it may imply update content was or may be installed that was not
generated by the official upstream.
Signed-off-by: Brad T. Peters <brad.t.peters@intel.com>
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This is a new sub-command for swupd called
"bundle-list", this command will show which
bundles are installed in the local system.
This information is obtained reading
/usr/share/clear/bundles path in local filesystem.
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
This function reads installed bundles from local standard
directory and store them in a list pointer passed as argument, this
function is the base for functionality as listing installed bundles to
user.
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
There is a need to override the cert path at runtime, so migrate to
using a global variable that is set in init_globals().
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
We change the telemetry API to pass a class record identifier,
which will be part of the telemetry record file name. This allows
us easier to send different telemetry class records to the server,
which will make parsing simpler in the probe and on the backend.
Rev record version to #2 because of this change.
The telemetry function only works insire swupd_init() / swupd_deinit()
functions, as it accesses state_dir from globals. This restricts the
use of it a bit, so we move the timekeeping function into
main_update().
Aside from update, I've added a telemetry point in verify as well.
With this, we have several easy telemetry points in swupd that
should give us an indication how swupd is performing without divulging
lots of detailed information.
The swupd.h file contains easy defines for level, and the telemetry()
function itself is as failsafe as it can be - any error results in
a continuing swupd program, although these errors would occur in
a system that is already hopeless (disk full) anyway.
The comment from commit fc0f570 was a bit misleading. Adding more
information about which combinations are valid and tested instead.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Now that the number of pending downloads is kept below a certain limit
(see poll_fewer_than()) it is possible to open files before starting
the transfer. Using the default curl write handler and explicit
open/close of the file makes the code simpler.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The previous approach was to open/fdopen/fclose the file for each
chunk that gets passed from curl. This incurrs a huge performance hit
when close() triggers a hashing of the file content on systems where
integrity protection via IMA is enabled.
Now the file is opened only once and kept open until the download is
complete. In addition, the unnecessary usage of C file IO is avoided.
The semantic is changed as little as possible:
- file gets created only after the first chunk of data arrived
- file descriptors do not leak to child processes (O_CLOEXEC)
- data gets appended to existing files (via O_APPEND, used
to keep the code simple and avoid an additional lseek)
- data gets flushed explicitly for each chunk (via fdatasync(),
which somewhat approximates the effect that an explicit
close() may have had)
As an additional improvement, failures during close() are checked. To
keep error handling as much as before, the completion function which has
the close() takes the current curl error code and replaces it if it
encounters a write error.
[v2 of the patch with fixes by Dmitry Rozhkov, see https://github.com/pohly/swupd-client/pull/1]
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Matthew Johnson