When running get_latest_version, the swupd_curl_get_file would allocate
a version_container struct on the stack that would then be used by the
curl callback to keep track of the buffer offset. This use was invalid
however because the callback was run after the stack had been popped
which lead to undefined behavior.
Instead change the swupd_curl_get_file function to take the struct
itself so it will refer to memory valid for the entire length of the
call.
The swupd_curl_get_file function should likely be restructured at some
point so that in memory downloads are less of a hack (only able to
download a LINE_MAX worth of data) at some point however and this will
need to be updated again.
Add new function consolidate_files that replaces
consolidate_submanifests in order to remove the requirement of modifying
manifest components in place. This change enables creating seperate
consolidated file lists for bundle-add, one for new files to be
installed on the system and one for the currently installed bundles
files. Once these have been seperated out, only files requiring
installation can be processed by do_staging and the list of files
installed on the system can be used to run verify_fix_path.
This also modifies bundle-add to stop trying to create tracking
files which was impacting testing.
In order to support verify_fix_path from bundle_add, a generic file list
from bundle list needs to be implemented to replace the one embedded in
the current consolidate_submanifests function.
This function will be used to generate a file list to be passed to a new
function for consolidating a list of files (created in the following
patch).
Note the list of files this creates is not sorted.
To support low duplication lists of files living in multiple locations
that will be required for bundle_add to be able to run
swupd_verify_fix_path add a list_clone function which will create a copy
of a list without duplicating the list data.
When pack is going to be downloaded an option for
curl called resume is activated no matter what,
however although major http server support 'Range'
command that enables this functionality, not all
complies with this, when swupd client tries to
re-download a partial file from one of the server
that does not support 'Range' it will just throw
an error and stop to work until partial download
is deleted by hand.
This patch address that issue by checking first
whether server supports 'Range' command and if not
then disable 'resume' of packs.
Signed-off-by: Jaime A. Garcia <jaime.garcia.naranjo@intel.com>
For safety, 'verify --fix' does not use swupd_rm() (like 'update') when
deleting files, etc. Instead, each deleted entry is considered
individually.
So for the deletion step to work correctly with 'verify --fix', files
contained within directories must be listed before the directories
themselves. A simple solution is to make the manifest reverse filename
sorted (opposite of the current filename sorted order).
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Since we are either determining the current OS version, or the latest OS
version, at any given time, rename these functions to reflect that.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Since most of these checks are used by all subcommands, and are written
in slightly different ways, move all of the checks into swupd_init() to
do the checks in one place and prevent misordering of these calls.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Instead of specifying a version number argument to -m allow passing the
latest string which will query for the latest version and use that as
the install target version.
This option overrides the global setup for state_dir
variable that is now used to find out where swupd
is going to do work (state dir).
If not --statedir option is passed along, the global
STATE_DIR definition value is going to be used instead,
this last one can also be changed at compilation time.
Signed-off-by: Jaime A. Garcia <jaime.garcia.naranjo@intel.com>
Similar to how the content URL, version URL, and format string globals
are now set, do a similar reorganization to set path_prefix in its own
function and update all subcommands to use this new function.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The intended behavior here is to check if the directory path for the
current file contains any symlinks, not the full path itself. By not
checking the directory path only, verify --fix would not delete any
symlinks that were marked deleted.
Also, because the directory path is being examined now, we can obtain
the directory file descriptor (dirfd) and use unlinkat(2) to delete
files and directories instead of unlink/rmdir; doing so avoids race
conditions in that paths may change between calling realpath() and
unlink().
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Previously include manifests that were not newer than the current
version would be silently ignored even if they had not been previously
installed causing bundles to miss included dependencies.
Fix this by adding a is_tracked field to struct file that is nonzero
when the bundle the file is contained in is marked as tracked via having
a /usr/share/clear/bundles/$name file.
The is_tracked == 0 version of the file will be dropped in the
consolidate_manifest phase if that filename from another bundle has
is_tracked != 0 to avoid downloading and installing an already existing
file.
The is_tracked == 0 version of the file will then be added to the update
list from the consolidated manifest if that file was not previously
installed on the system (in addition to the other cases it would already
have been added to the list for).
Because combinations of using the libcurl FILE protocol and HTTP/HTTPS
are not supported, make sure that the two protocols specified for -c and
-v match.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Instead of hardcoding the default version URL and content URL in the
binary, read the defaults from config files that live in
/usr/share/defaults/swupd/.
If the config files do not exist, then users can pass the -c or -v
options as necessary (or -u, which sets the same value for -c and -v).
The primary motivation for this change is to enable a better experience
for the swupd mixer; users mixing a version of Clear Linux can set the
default URLs at mix time instead of always passing -c/-v/-u at runtime.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Add support in update to properly support bundles that include other
bundles inside them. This feature is intended to work with bundle
additions by adding included bundles (from bundles already subscribed)
to the subscription list prior to downloading packs and full files.
As part of this change the add_subscriptions function from bundle.c was
modified to handle retrying manifest downloads.
Instead of taking in versions as arguments to download_subscribed_packs,
just use the version information in the subs list items (set by calling
subscription_versions_from_MoM function).
Because the subs list may contain bundles that are new in the version
being updated to, don't set an error in the subs item for things not in
a particular manifest version.
The latest_version in check_versions and read_versions was a legacy
variable used for btrfs support that no longer exists and was a
duplicate of current_version in all use cases. This patch removes the
variable completely from the above functions.
Since GNU tar fails to extract files with xattrs preserved when
Integrity Measurement Architecture (IMA) is enabled some vendors
may choose to install libarchive-based tar (bsdtar) on their embedded
devices, so the swupd server needs to be able to create archives
in its format.
This patch adds one compile-time options --enable-bsdtar that is used
to enable/disable GNU tar specific options.
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
Eventually manifests will be able to include other manifests and this
patchset adds support for adding those included manifests when running
bundle-add on the including manifest.
Start handling the case where a manifest has an includes line. This line
will be used to specify other manifests that are to be installed with
the containing manifest.
Pull out code that only can be run once into a bundle installer frontend
function and move code that could be run recursively into an another
function. This is done in preparation for manifest includes.
A user was running into issues installing the kernel-container bundle on
Clear Linux. Namely, all missing files marked as boot were not
installed.
The root issue is that the bundle-add action does not set the "fix"
global variable before calling ignore(), so ignore() mistakenly returns
true when considering boot files that are missing, but not marked
deleted.
Because 'bundle-add' should be installing new boot files, and 'verify
--fix' should be repairing boot files if they have mismatching hashes,
remove the global "fix" variable entirely, since in all current call
sites, the "fix" behavior is expected.
Also, add another call to ignore() in remove_orphaned_files() to ensure
that any file that should be ignored (including deleted boot files) are
not removed during a 'verify --fix'.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
To support swupd-client testing without connecting to a web server, it's
convenient to use libcurl's FILE protocol, which will query files on the
local filesystem instead.
Because the libcurl-multi download path for fullfiles has treated the
"0" response code as an error, but it has the opposite meaning when
using the FILE protocol, only set the error when connecting to a web
server.
Also, the libcurl-easy code for synchronous downloads didn't treat "0"
specially, so this commit ensures the proper handling, as well as
setting a generic download error for the FILE protocol specific
CURLE_FILE_COULDNT_READ_FILE.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The verify_fix_path function takes a path and a consolidated MOM as its arguments.
It breaks down the path into subpaths and checks if each subpath is missing.
If found missing, the path is searched for in the consolidated manifest for its hash
and downloaded synchronously. It returns success if all that subpaths are verified
and fixed.
The following command was run with clang 3.7.1:
$ clang-format -i -style=file include/*.h src/*.c test/*.c
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
New swupd primary command, offers the ability to
search for the provider of a specified binary or library.
A full complement of bundle manifests for the os-release version
is downloaded to the client prior to conducting any search. This
patch has been designed to enable this download, or staging, to
take place prior and separate from use of the search function.
However, with each search, the manifest complement is checked to
ensure completion.
Overhead:
A check against the Clear Linux 6300 manifest set
shows a search will entail a ONE TIME 83 MB network download
(the compressed manifest set), and a constant decompressed 400MB
usage on disk. Future searches will only require new or modified
manifests be downloaded.
Finally, download size is provided to notify user of expected delay
-----
Ex:
"Downloading manifests. Downloading 83.23 MB..."
Signed-off-by: Brad T. Peters <brad.t.peters@intel.com>
William Douglas