When running verify --fix with -m specifying a different version than
the current OS version, require --picky to enforce management of files
under /usr or require --force as an override. Adding --picky can solve
issues such as issue #238 where fixing backward caused glibc library
confusion.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Allow user to list all bundle dependencies of a bundle passed to
swupd bundle-list --deps BUNDLE
This lists all included bundles (included those recursively included)
for the BUNDLE. This is particularly useful when applications or users
need to determine just how large the bundle addition will be. For
example, a user may not be aware that by bundle-adding desktop-dev they
are recursively adding 12 other bundles. This command allows the user to
make this check before installation.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
The --has-dep=BUNDLE argument will display a tree representing all
installed bundles that recursively include BUNDLE.
One may pass the --all argument as well to list the dependency tree for
BUNDLE including all installable bundles available on the server.
**Example output without --all:**
Installed bundles that have os-installer as a dependency:
format:
# * is-required-by
# |-- is-required-by
# * is-also-required-by
# ...
* mixer
|-- os-clr-on-clr
**Example output with --all:**
Attempting to download version string to memory
All installable and installed bundles that have os-installer as a dependency:
format:
# * is-required-by
# |-- is-required-by
# * is-also-required-by
# ...
* clr-devops
* mixer
|-- os-clr-on-clr
|-- os-clr-on-clr-dev
|-- clr-devops
|-- os-clr-on-clr-dev
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
When using verify's -m option, it generally only makes sense to verify a
build within the same format, since the subsequent format (if any) might
not be compatible. Therefore, we should disallow verifying to a build
with a different format by default. This commit adds the proper
enforcement by making this condition a fatal error. The early exit can
be bypassed using the -x/--force option, which will print a warning
message instead.
For now, this compatibility check is targeting the use case of verifying
to a newer build, so I also report the latest supported build for the
current format. Verifying to an older build is not guaranteed to succeed
at present, and swupd-client does not yet understand the
version/formatN/first file. In other words, reporting to the user the
oldest build they can verify to will be a future improvement.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Add a basic progress indication for a couple potentially long-running
swupd tasks (download_pack and add_missing_files). The progress
indication is displayed as a percentage:
...85%
It overwrites its own line so the progress will update in place. It may
be interrupted by other swupd output:
...45%
Extracting os-core pack for version 10000
...50%
Extracting os-core-update pack for version 9000
...55%
The progress indication will then resume updating in place, on the same
line.
This progress is only displayed when outputting to a TTY, and will not
output anything when the output is being redirected. This is nice for
testing and logging, but means that these progress updates will not be
caught by processes that capture swupd output, such as ister.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Instead of allowing only one "actions:" field in the Manifest.MoM, allow
for several and read each into the post_udpate_actions list. This allows
us to add more actions down the road if desired.
Currently the only use for the "actions:" field is to indicate when a
re-update is required. This is handled by checking if the string
"update" is in the post_update_actions list.
Finally, remove the warning to the user to perform the post update
action themselves, as this will be handled by swupd itself. Functional
tests updated to reflect the missing output.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
Adds a check to compare version numbers in <state_dir>/version and
<target_dir>/usr/lib/os-release when attempting to re-update after a
format bump.
os-release should always be equal to <state_dir>/version for normal
updates and potentially greater than <state_dir>/version for format
bumps.
In the event of a bad build where os-release does not get updated to the
new version, this check will prevent an infinite loop where swupd reads
the current version from os-release, sees it is out of date, attempts to
update again, but the file does not get updated.
This occasion is very unlikely but the guard is nice to have
re(guard)less.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
When a device is out of date by more than a format bump re-exec swupd
update until the device version matches the server version. This is
achieved by reading the "actions" field in the Manifest.MoM, which is
yet to be implemented in swupd-server. When "update" is specified in the
actions of the current manifest and the current update is successful,
swupd will re-exec itself with the same flags it was originally called
with.
Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
We draw a hard line with error codes, such that:
Error code 16 is reserved for failures resulting from the basic
network check. Any call to check_network() that fails will
return this error code. No other error path returns this
error code.
Additionally, check_network() is called in every normal code path.
All other, possibly network related issues, return a different
error code. If the basic network check succeeds, but e.g. pack
downloads fail, we return a new (23) error code so that we
can better establish the conditions through telemetry and
determine whether the error is on the client or the server,
which is highly likely with this new error code 23.
This is the basic implementation for a "factory reset" type of command.
It will take the functionality of verify --picky and act on the files found
as verify --fix does with files found mismatching in the manifest.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Add a "swupd verify --picky" command which lists files under
/usr (modified by the --path option) which are not listed in the
current manifest. Exceptions /usr/local and /usr/lib/modules. The
former to allow for local changes, the latter because the clear boot
manager owns that directory.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
The update process should only read in post update action from the newest
MoM. This makes it so that even with a previous MoM having an action, only
the new one will be taken, and if none exist, the old one will be ignored
so swupd will not take the same action as the previous update.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The swupd client should support parsing extra data in the case that a format
bump occurred, or other possibly breaking change, so that it can at least
notify the user of more action needing to be taken post update.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current implementation is not setup to optimally load manifests and attempt deltas.
This changes the flow to be as follows:
1.) Load current Manifest.MoM
2.) Load server Manifest.MoM
3.) Recurse and load all current manifests, since we know exactly which are needed already
4.) Link current and server Manifest.MoMs, setting versions and peers
5.) Load server manifests recursively adding their includes
6.) Recurse the server manifests to find and load any new included bundles
7.) Link current and server Manifest.MoMs again to account for new bundles
When loading the server manifests, deltas will attempt to be applied to
current manifests before doing a full download. Following this flow enables
swupd to load manifests in a logical order, and short circuit trying to
reload them later on as manifests are recursed for includes again later.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The current version must be passed to add_subscriptions, else it will call
load_manifest() with equal current and server versions and never enter the
try_delta_manifest code path.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
We must not run these with --no-block because order must be preserved,
and critical components may need to be reexec'd, making this a core piece
of the swupd update process.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
For various reasons - partial downloads, power failure, user exiting -
the /var/lib/swupd state directory may become corrupt. When this happens,
manifests can be rendered unusable, causing various errors in swupd such as
signature failures, or errors reading the manifest. This patch attempts to
avoid such things causing updates or verifies to fail by cleaning up the
Manifests artifacts of what it could not use, and re-downloading them to
try again.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This introduces a -t/--time option to update, verify, and bundle-add commands, displaying raw elapsed time and CPU process time for integral swupd operations within the given code paths.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The system clock may be terribly off, especially on new hardware that has not
yet been calibrated. Updates rely on the certificate and system time being
sane to verify validity, so if a mismatch is found the certificate will
be deemed invalid and the update stopped. This patch attempts to fix the
system time to something sane using the time from the swupd binary itself,
which should not have been touched by any user except root. If the time is
normal and verification fails, the cert cannot be trusted.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This sets swupd signature verification to mandatory, failing and exiting
for any verify error, though --nosigcheck enables user to bypass all
signature-based security checks.
The signed MoM serves as the top level chain of trust, and it is used to
extend content trust down to the individual file level. When the signature
of the top-level MoM is invalid or cannot be verified for any reason, we
warn and abort the operation.
Passing --nosigcheck will allow this to proceed, explicitly accepting the
unverifiable MoM and outputting a log entry to the Journal. This is not
recommended and unsupported by upstream once the chain of security is broken,
because it may imply update content was or may be installed that was not
generated by the official upstream.
Signed-off-by: Brad T. Peters <brad.t.peters@intel.com>
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This is a new sub-command for swupd called
"bundle-list", this command will show which
bundles are installed in the local system.
This information is obtained reading
/usr/share/clear/bundles path in local filesystem.
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
This function reads installed bundles from local standard
directory and store them in a list pointer passed as argument, this
function is the base for functionality as listing installed bundles to
user.
Signed-off-by: Mario Alfredo Carrillo Arevalo <mario.alfredo.c.arevalo@intel.com>
There is a need to override the cert path at runtime, so migrate to
using a global variable that is set in init_globals().
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
We change the telemetry API to pass a class record identifier,
which will be part of the telemetry record file name. This allows
us easier to send different telemetry class records to the server,
which will make parsing simpler in the probe and on the backend.
Rev record version to #2 because of this change.
The telemetry function only works insire swupd_init() / swupd_deinit()
functions, as it accesses state_dir from globals. This restricts the
use of it a bit, so we move the timekeeping function into
main_update().
Aside from update, I've added a telemetry point in verify as well.
With this, we have several easy telemetry points in swupd that
should give us an indication how swupd is performing without divulging
lots of detailed information.
The swupd.h file contains easy defines for level, and the telemetry()
function itself is as failsafe as it can be - any error results in
a continuing swupd program, although these errors would occur in
a system that is already hopeless (disk full) anyway.
The comment from commit fc0f570 was a bit misleading. Adding more
information about which combinations are valid and tested instead.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Now that the number of pending downloads is kept below a certain limit
(see poll_fewer_than()) it is possible to open files before starting
the transfer. Using the default curl write handler and explicit
open/close of the file makes the code simpler.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The previous approach was to open/fdopen/fclose the file for each
chunk that gets passed from curl. This incurrs a huge performance hit
when close() triggers a hashing of the file content on systems where
integrity protection via IMA is enabled.
Now the file is opened only once and kept open until the download is
complete. In addition, the unnecessary usage of C file IO is avoided.
The semantic is changed as little as possible:
- file gets created only after the first chunk of data arrived
- file descriptors do not leak to child processes (O_CLOEXEC)
- data gets appended to existing files (via O_APPEND, used
to keep the code simple and avoid an additional lseek)
- data gets flushed explicitly for each chunk (via fdatasync(),
which somewhat approximates the effect that an explicit
close() may have had)
As an additional improvement, failures during close() are checked. To
keep error handling as much as before, the completion function which has
the close() takes the current curl error code and replaces it if it
encounters a write error.
[v2 of the patch with fixes by Dmitry Rozhkov, see https://github.com/pohly/swupd-client/pull/1]
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Swupd used to use a sorted blob of extended file attributes as part of
the data that was used to verify the contents. This caused problems if
extra attributes were added. In particular if a clearlinux system was
being run in a container that had selinux enabled in the base OS.
Add a configure option to allow the attributes to be considered or
not. Based on advice, the default is to ignore them, as this is what
everyone expects.
Tidied up passing a --selinux flag to tar, this is also handled by a new
config option.
Signed-off-by: Icarus Sparry <icarus.w.sparry@intel.com>
We spend a lot of wasted time parsing the entire manifest in some areas, when
only the header is needed. This patch adds a header_only flag that tells the
appropriate load manifest functions to quit early after the header is read,
instead of parsing the entire (possibly very large) manifest. Another
optimization is calling fopen() with the 'm' flag, which tries to use mmap to
access the file (for reading only).
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This patch cuts down the amount of recursion that happens by skipping a
subsequent rescursive calls to add_subscriptions if we hit a bundle that is
already subscribed.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Instead of printing the stats for what changed overall in the MoM, it's
more interesting to report what changed on their system. That is,
reporting which bundles are new, have changed, or (in the future) have
been deleted.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
To avoid calling the version-setting function multiple times for a
single subscription list for 'update', simply reference both MoM manifests
in the function. For other subcommands that only reference a single MoM
manifest, the second argument should be NULL.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
A forthcoming commit will make 'update' understand two subscription
lists, one each for the current and latest versions, so using a single
global list will no longer be sufficient.
Instead, allocate a subs list for each subcommand that needs it, and
adjust helper function signatures that need to read/write the subs list.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Updates the order of preference for content_url, version_url
and format_string, from:
1. Runtime flags
2. Configure time options
3. State dir files
to:
1. Runtime flags
2. State dir files
3. Configure time options
This patch also changes the logic of the setter functions to allow
multiple calls. Once the respective global is set, that function will
return true or 0, ie "success", (depending on the function).
Signed-off-by: Brad T. Peters <brad.t.peters@intel.com>
Remaining hits by findstatic.pl are public API functions and
dangling statistic increments (but those last are fixed by the
PR cleaning up the stats, so they can be ignored for this PR).
This is a negligable performance increase and more of a cleanup
of symbol space. In order for these to inline, we need the stats
to be exported instead, so we go from 8+1 exports to 1+1.
Since signature initialization occurs in swupd_init(), use the same
convention as other init steps by defining/using a special error code.
As a side effect, the return value type is fixed (int vs bool) for the
signature init failure case.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
To make compile-time options more easily discoverable at runtime, make
the --version output more verbose by printing a few of them. Other
options can be easily added in the future by adding an appropriate
AC_DEFINE call in configure.ac, and updating the new header added in
this commit (swupd-build-opts.h).
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Calculate the hash for directories is desiderable to be independent
on the dirname due to the subsequent calculation on the staged/HASH
file. Here is used const "DIRECTORY" string for input name for
all folders.
Signed-off-by: Jose R Guzman <jose.r.guzman.mosqueda@intel.com>
The poorly documented libcrypto in OpenSSL apparently does not like
its initialization and cleanup functions to be called but at process
start and exit. Swupd-client already has a swupd_init() so easy on
that side. But there was no common swupd_deinit() in which to place
terminate_signature(). I add one and put the common exit cleaners in it
and fix of a number of little inconsistencies around process exit cleanup
that have come to exist because there was a common cleaner.
With this, the signature verification of both the current and latest
MoM works, where prior only a first verification succeeded.
This patch also fixes a few memory leaks, though there are still many
memory leaks in swupd bundle-add and bundle-remove, unrelated to signature
verification. And swupd search is segfaulting due to how load_mom()
is introduced in 8cf0ef91dd.
Signed-off-by: Tim Pepper <timothy.c.pepper@linux.intel.com>
This patch adds functionality for the swupd client to do signature
verification on the MoM, which ensures a root of trust for the rest
of the update by guaranteeing the authenticity of the MoM and content
it includes.
As we focus now on just verifying the signed MoM, a number of functions now
become static to src/signature.c.
By default MoM signature verification is disabled. Configure
--enable-signature-verification to enable it. When enabled the default
cert for verification is /usr/share/clear/update-ca/ClearLinuxRoot.pem, as
specified by concatenation of SWUPDCERT onto UPDATE_CA_CERTS_PATH. The
SWUPDCERT can be overridden via configure --with-swupdcert=some.pem.
When signature verification is enabled, and the MoM's signature does _NOT_
verify, currently only a warning is presented but the swupd operation
continues. In the future signature verification will become mandatory.
We first need to sort out a few details with mixer to insure the right
thing happens there.
Signed-off-by: Tim Pepper <timothy.c.pepper@linux.intel.com>
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
In order for the chain of trust rooted at the Manifest.MoM to operate
correctly, the bundle manifest hashes (as listed in the MoM) must be
checked. For now, warnings will be emitted when hash checks fail, but
they will become errors in the near future.
This commit simplifies/splits the load_manifests() interface into
load_mom() and load_manifest(), since load_manifests() was becoming too
complex, since the handling needs for MoMs versus bundle manifests is
sufficiently different. For the new load_manifest(), the logic is
changed to first download the manifest, then check the hash, and only
then load the manifest into memory.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Matthew Johnson