This release contains various changes to fix segfaults and memory misuse,
simplifying logic when iterating directories and populating file structs
from manifests, and changing the secondary sort to be lexographically sorted
filenames for version-sorted manifests.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release contains various changes to fix segfaults and memory misuse,
simplifying logic when iterating directories and populating file structs
from manifests, and changing the secondary sort to be lexographically sorted
filenames for version-sorted manifests.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
To ease human readability of manifests, but without impacting manifest
delta efficiency, use a lexicographic filename secondary sort order when
sorting manifests by version.
Below is an example of how this commit changes the sorted order (the
first column is the version, and the second column is the filename).
# Before
10 zyxw
10 abcd
20 test2
20 test1
20 abba
20 aaaa
30 zzyy
# After
10 abcd
10 zyxw
20 aaaa
20 abba
20 test1
20 test2
30 zzyy
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The file struct must be populated to contain the proper stats, so just
check if it is a directory and iterate, ignoring the case where d_type may
not be defined on the system.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
According to POSIX.1 only d_name and d_ino fields of struct
dirent are standardized. d_type isn't always correctly set on file
systems like XFS. In such cases it makes sense to resort to
lstat(). Otherwise a user has hard time figuring out what's
wrong with her setup.
Also remove redundant populate_file_struct() as it's called
again in parallel threads.
Signed-off-by: Dmitry Rozhkov <dmitry.rozhkov@linux.intel.com>
If the os-core bundle is not listed in groups.ini we will run into
problems later on, including segfaults when processing bundle includes.
Check the os-core bundle is listed in the groups.ini during initialisation
and error gracefully when it is not.
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
In the (unlikely) case that nothing changed between two builds,
get_deduplicated_fullfile_list() segfaults because it uses
manifest->files without checking for NULL, aka the empty list.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
At the moment, swupd_create_pack fails when some files have xattrs and
get patched because the xattrs of the test file do not match the
original, unpatched file.
That's because xattrs_copy() was applied to the wrong target file.
Fixes: swupd-server/#35
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
The bundle-chroot-builder changed the location where it stores bundle
includes metadata, so swupd-server needs to read from the new location.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
This patch extracts the [STATUS] field from our bundle information
metadata, and stores it in groups.ini. From there we put the contents
of this string verbatim into the manifest.
We don't interpret, encode or convert the contents of the [STATUS]
field in the manifest. Instead, we just strip non-alphanumeric
characters and pass the contents on. This leaves it entirely to the
client to parse and interpret the value of this field in the manifest.
If the bundle file, or the groups.ini file omits any status, nothing
is output to the Manifest file.
This release introduces a change to the hashing scheme regarding delta names,
and directory hashes, as well as minor fixes to support the new hashing.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
There is an issue when two different files in a newer release have same hash:
This is, when swupd updates a file by applying a delta, it takes the HASH2 to
know the file where delta must be applyed. If files are different in current
release (before updating), there must be 2 deltas, one for each file but as
the two files have same hash in new release delta's name are the same:
FROM-TO-HASH2 and it is when issue arises due to just the last delta file is
kept when swupd server creates files and packs. So when swupd client tries to
apply the delta to one of the file that does not corresponds it will fail and
generates an error. At the first look it will seem like delta file is corrupted
however the issue is that delta file was created for another file.
To solve this issue we include the hash for the original file in the delta's
name so that swupd client can take the correct delta and apply it:
FROM-TO-HASH1-HASH2.
Signed-off-by: Jose R Guzman <jose.r.guzman.mosqueda@intel.com>
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Calculate the hash for directories is desiderable to be independent
on the dirname due to the subsequent calculation on the staged/HASH
file. Here is used const "DIRECTORY" string for input name for
all folders.
Signed-off-by: Jose R Guzman <jose.r.guzman.mosqueda@intel.com>
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release includes a fix to replace system calls by local system_argv
functions, cleanup patches for error output and code style, adding delta
manifests to delta packs, consistency changes for tar and string allocation,
and removes creation of delta manifests out of create_update.
This is a re-roll of release 3.2.4 to fix a time skew issue that caused some
build failures.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release includes a fix to replace system calls by local system_argv
functions, cleanup patches for error output and code style, adding delta
manifests to delta packs, consistency changes for tar and string allocation,
and removes creation of delta manifests out of create_update.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
Delta manifests are handled by the pack creation step, and should not be
part of the plain manifest/update content creation step. This reduces the time
swupd_create_update takes to run significantly.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This reverts commit 7f3b760430 and commit
74f4afa655. The code introduced parallel manifest
creation, and removed delta manifests from being created, but needs some more
rigorous testing and refactoring to ensure all test suites pass.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
I had neglected to rebase to the latest revision of #21, so this commit
incorporates the additional changes.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The code previously created all the bundle manifests and delta manifests
synchronously in 1 thread. While this worked, it was not optimal considering
the manifests can be created at the same time since they are unique. This
patch creates a threadpool that processes each bundle manifest in its own
thread. Delta manifest creation was removed from create_update because it
can already be handled by our delta pack creation process, and it is a huge
time-sink for create_update as it also runs single threaded.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
The delta manifests were being staged correctly but were not being added
to the delta packs.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
There are new functions that replace system() calls: system_argv(), system_argv_fd and system_argv_pipe().
They are based on execvp() function and they can manage any argument including filenames with characters
that could be interpreted as meta-characters in a regular system() function.
So no escape for the arguments needed and is less prone to errors.
Signed-off-by: Jose R Guzman <jose.r.guzman.mosqueda@intel.com>
Since delta pack creation is very much tied to a DevOps workflow, it
makes more sense for the pack_maker.sh to live outside of swupd-server.
This script will move to the clearlinux/mixer-tools repo, since the
Clear mixer is a DevOps-oriented tool.
Also, update the code comment in versions.c to describe another action
to be taken that is DevOps-specific, namely that the value of
WEBDIR/version/formatN/latest file should be set at release time.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
This release fixes a bug with adding invalid bundle manifest hashes to
the Manifest.MoM and optimizes swupd_create_update to remove the
unnecessary rsync from the full chroot back to bundle chroots.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Just in case we encounter a bundle manifest that does not exist anymore,
or the last_change is 0, which is an invalid version to create, make
sure we add the deleted entry in the MoM and do not attempt to untar
that manifest.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
The bundle manifest hashes were calculated from the STATE_DIR Manifest.bundle
files, which caused a problem when they the permissions/owner were changed. The
hashes should be calculated by untarring the Manifest.BUNDLE.tar files
referenced in the Manifest.MoM.
Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
This release includes a trivial improvement for reporting paths that
contain blacklisted characters.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Tudor Marcu