2146 Commits
Author SHA1 Message Date
William Douglas d68bc6fb83 Release v5.0.0
- Major change in how manifests are parsed.

The manifest's modifier flag has been reworked, previous flag settings
are now ignored and they get the default behavior of the new
settings. Only default modifiers are currently supported, all other
types are skipped at this time.

Signed-off-by: William Douglas <william.douglas@intel.com>
v5.0.0
2023-03-02 12:39:32 -08:00
William Douglas 77f16b0120 Add new handling for modifier field
The modifier field will be a character that translates roughly into a
base64 encoded bitmask. The first 3 bytes are the optimization level
and the last three are the max optimization level any files with that
same filename can have.

For now though skip all lines that don't match SSE_OPT (the previous
values the modifier field would have: 'b', 's' and 'C' all map to
the SSE_OPT bitmask).

Also rework the manifest unit tests a bit more. Noticed the exported
flag was wrong in the data and the X flag for the 3rd field was
completely unchecked for in swupd code. Added testing for the ignored
'b', 's' and 'C' values of the modifier field and checked the non-sse
is in fact skipped as a file for now.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-03-02 12:35:32 -08:00
William Douglas bf3b0201c3 Fetch entire git tree for master comparison
The api test needs to compare the current commit on the remote branch
to the master branch. Based on the documentation, 'fetch-depth: 0' is
intended to fetch all branches and tags.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-28 13:25:05 -08:00
William Douglas ca3b07cf5e Update to use latest checkout action
Update all workflow's checkout action to the latest version.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-28 13:25:05 -08:00
William Douglas a011f9429a Remove logic for flag not in use
The logic is testing a flag that mixer does not set and would like
to change the purpose of in a new version. Remove the test (needs a
format bump before mixer can use the flag) for the flag.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-28 13:25:05 -08:00
William Douglas 6896d8f155 Rework output checking in test
The disk space check was trying to match exact number of download
attempt messages which was not working consistently.

Move to simply detecting the failure messages once.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-22 00:04:12 -08:00
William Douglas 98a3362ae9 Rework clear linux docker container for test runs
Also skips a few tests that do not run well with docker (changing date
and disk filling have a hard time). Adding comment for how to run
docker (needs the --cap-add LINUX_IMMUTABLE option passed).

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-22 00:04:12 -08:00
William Douglas a9608851ee Move packs to download synchronously
There are some problems with how we are using multiplexed curl causing
huge slowdowns (2-5 times slower).

For now rather than rework the multiplexed curl code, switch to using
the synchronous code path.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-21 23:26:19 -08:00
William Douglas 3338cb2c66 Rework clean-old-statedir test output checking
The order of files in this test aren't very completely stable so make
do with looking at output chunks that seem to be reasonably consistent
across different environments.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas 7f59817a43 Improve tests for container
Detect docker container usage when running test to handle cases where
systemd is unlikely to be running.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas 776f8c92b0 Update use bats setup and teardown features
Instead of relying on a custom global_setup and global_teardown
functions, migrate to the use of bats' setup_file and teardown_file
functions.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas eddf0dc2a4 Fix shellcheck SC2048 error
Correctly quote arrays.

Also improve some commands getting the array content (fixes space
issues with quoting).

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas 6a6a907b06 Move no space tests to CI only
These tests fill up the disk and can be quite annoying to run on a dev
system normally.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas e55e33afef Fix shellcheck SC2269 error
Remove useless assignment.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas ffb1590909 Update shellcheck exceptions
Shellcheck changed its detection for a few file sourcing errors from
1090 to 1091. Update the disables accordingly.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas bf5e1f1297 Fix missing header
Header file ordering hid a missing list structure definition.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas 436fb503c6 Don't version clang-format
clang-format changing output between versions is painful so might need
to be reconsidered at some point.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas dbeb3662a4 Apply changes from clang-format
Running:
clang-format -i -style=file src/*.[ch] src/lib/*.[ch] \
src/3rd_party/*.[ch] src/cmds/*.[ch] src/swupd_lib/*.[ch] \
src/verifytime/*.[ch]

with clang v15

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas a425393c87 Update cache action
Old action was using deprecated API.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
William Douglas 52ae6dadc1 Update rst2man path from Ubuntu
Correct the path where the rst2man script lives for CI to find it.

Signed-off-by: William Douglas <william.douglas@intel.com>
2023-02-17 12:58:05 -08:00
carlwiede 2341efda7e Fix broken links 2023-02-07 14:07:18 -08:00
William Douglas 7e00e0a20b Switch off of string n functions
These functions lead to spurious errors and warnings for our use
cases.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-10-07 12:51:05 -07:00
William Douglas b6a4aff5b7 Remove deprecated function use
Since OpenSSL 1.1.0 this function is no longer needed.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-10-07 12:51:05 -07:00
Arjan van de Ven 4774daf00d allow for non-versioned delta files 2022-10-07 12:51:05 -07:00
William Douglas 49adbbd18f Add additional shellcheck directives
Signed-off-by: William Douglas <william.r.douglas@gmail.com>
2022-10-06 14:34:40 -07:00
Brett T. Warden 0e0cc9b969 Update offline installation tests with URL override
Offline installation tests also override the installation URL, so we
must update the expected output to contain the new message about URL
override.
2022-10-05 11:08:47 -07:00
Brett T. Warden db4ceb85b8 Update tests for swupd mirror commands
Mirror tests specifically invoke the URL override options that trigger
the new text output I've added, so the tests' expected output needs to
be updated to match.
2022-10-05 11:08:47 -07:00
Brett T. Warden 39515d2fc0 Announce overrides of content and version URLs
When using -v, -c, or -u to set alternate version or content URLs, print
out a message so it's more obvious.
2022-10-05 11:08:47 -07:00
William Douglas 3d611782ab Move to a static pemfile
There is something going wrong with installing the generated pemfile.
The file is created but not yet valid, this is not a problem for
local testing for some reason. To work around this issue, use a static
pemfile that is already valid and force install it.

Signed-off-by: William Douglas <william.r.douglas@gmail.com>
2022-10-04 17:25:31 -07:00
William Douglas dbfce4b218 Skip the timesyncd stop and start
As these are CI only tests, the container image currently appears to
have the service masked so stop trying to modify it.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-10-04 17:25:31 -07:00
William Douglas 685ed470c4 Use regex match for size comparison
The exact size difference changed for some reason but shouldn't cause
the tests to fail as it isn't the point of this test.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-10-04 17:25:31 -07:00
William Douglas e778dc2ede Update test based on size
The test requires an exact size reporting, fix the size but consider
using a regex match instead.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-10-04 17:25:31 -07:00
William Douglas ca49f63feb Update CI scripts
Update CI scripts in attempt to get CI working with the current Ubuntu
container.

Signed-off-by: William Douglas <william.douglas@intel.com>
2022-10-04 17:25:31 -07:00
Mark D Horn 8b7525ec87 Fixed typo in man page.
Fixes: https://github.com/clearlinux/distribution/issues/2189

Signed-off-by: Mark D Horn <mark.d.horn@intel.com>
2020-11-10 12:06:16 -08:00
Castulo Martinez 935053fdbe Testlib: refactoring setup to load correctly
The setup() function is run at the beginning of each test by the BATS
system, we leverage this function to implement a global_setup() and a
test_setup().

This commit refactors the function to make it easier to understand, also
makes sure it loads the environment variables correctly for tests.
2020-06-16 11:42:13 -07:00
Castulo Martinez 838022f36b Testlib: making variables available to all tests
Variables are available just after being exported, however variables are
not kept between tests since the process that orchestrate the tests of a
test file is a parent process and child processes cannot set variables
of a parent process. This is problematic because we were loosing some
variables (like all third party variables) when running multiple tests
using a global_setup().

This commit fixes the issue by making all the environment variables
available to all tests in a test file regardless of if they use
test_setup() o global_setup().

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-16 11:42:13 -07:00
Castulo Martinez 239376c425 Testlib: adding new setters for env variables
Unfortunatelly not all environment variables can be set in the same
place, since some values may depend on a specific function being run
before. However it is important to have easy to lacte places where env
variables are being set.

This commit adds a few more setter functions for envrionment variables
so they are easily identified.

Closes #1489

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-16 11:42:13 -07:00
Castulo Martinez da83141022 Testlib: set variables from a common place
Some env variables in testlib can be exported when the file is sourced,
but some other have to be exported only after the test environment has
been created because they depend on it and in values gotten from it.
However testlib should try to set env variables in as little places as
possible.

This commit moves the definition of variables that can be defined in
set_env_variables() to consolidate them there if possible.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-16 11:42:13 -07:00
Castulo Martinez 7dd6fc221a Testlib: renaming variables for consistency
Environment variables are used everywhere in testilb. This environment
variables are global variables that define the way testlib behaves.
However is was confusing to use the variables because they were
inconsistent between each other, for example some variables that define
paths would have absolute paths while other would have relative paths,
making it error prone while using them.

This commit makes the environment variables more consistent by following
a name convention for each type of variable, as an example, variables
that define absolute paths follow this convention ABS_<path_name>_DIR,
while variables that define relative paths are defined like this
<path_name>_DIR.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-16 11:42:13 -07:00
Castulo Martinez a48ff926f3 Testlib: adding one more env variable to the help
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-16 11:42:13 -07:00
Castulo Martinez ac837cb9fe Testlib: Replace zero_hash constant with ZERO_HASH
Global constants in testlib are just exported variables, so they should
be in all upper case so we know they were defined globally.

Closes #1484

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-16 11:42:13 -07:00
Castulo Martinez 47a2fc08d5 Testlib: adding info about BATS variables
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-16 11:42:13 -07:00
Castulo Martinez 04dfaa1db7 Allow paths that contain symlinks for cachedir
When the path provided by a user for the --statedir (or --cachedir)
contains a symlink swupd will fail to extrar tar files since
libarchive does not support extracting files through symlinks.

This commit fixes the issue allowing users to use symlinks in the
--statediri/--cachedir path.

Closes #1581

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-15 15:24:34 -07:00
Castulo Martinez cdf03c14d4 Testlib: include important env variables in help
After sourcing testlib, if the user runs testlib from the terminal,
the help of the library is displayed which includes the list of
commands provided by the library.

Testlib uses some environment variables to modify the way tests are
run. This commit adds information about those env variables to the
testlib help.

Closes #1479

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-10 12:11:02 -07:00
Castulo Martinez defd541061 Fixing a false positive in a "clean" test
When checking the output of the "clean --dry-run" command in the test,
we expect the output to be shown in a specific order, but the order can
change depending on the system since readdir doesn't warranty a specific
order when reading the content of a directory.

This commit fixes the test by only checking for the files we are
intereseted in, and it checks them with separate assertions to make sure
we don't have false positives due to the order.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-08 15:40:29 -07:00
Castulo Martinez 68b4eee984 Adding flags to set data and cache in swupd
This commit provides the --datadir and --cachedir flags that can be used
to specify different locations for the data and cache. If the same
location is desired for both (default) then users can still use the
existing --statedir flag.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-08 12:10:39 -07:00
Castulo Martinez 40472f5a21 Updating "clean" to remove new and old cache dirs
"Swupd clean" removes cache from certain pre-defined directories. This
commit udpdates the locations where clean looks for cache so it includes
the new locations where the cache is stored.

Swupd will continue looking at old cache locations until the bump to
format 31.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-06-08 12:10:39 -07:00
Castulo Martinez 8d54e6c995 Separating the cache from the data used by swupd
Currently there is only one location, known as "statedir", where swupd
stores data and cache. Normally it make sense to keep the data in a
path that is relative to the path prefix of the target system, since the
data is specific to that system, but cache can be independent of the
system, and that way it can be reused for other target systems if
desired.

This commit splits the cache and data into two different locations.
It also stores the cache in a path that is dependent of the mirror url
being used by swupd. The commit also makes all references to files in
the cache or data directories through getter functions thus removing the
hardcoded reference to the file's location, making it easier to move the
location in the future if needed. Lastly, this commit changes the read
permissions for the data directory and the manifest directories in the
cache to be user readable.
2020-06-08 12:10:39 -07:00
Otavio Pontes 91173f50b2 heuristics: Use a sorted list algorithm to apply heuristic rules
Instead of comparing all rules to all files we can use the fact that the
file list is already sorted to compare only the elements that we need.

So if n is the number of files and m is the number of rules we moved from a
O(n*m) code to O(n+m).

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-05-29 14:42:54 -07:00
Otavio Pontes 99823e0d66 sys: Use a list for get_mounted_directories()
Instead of saving it in a global string, just use a list of strings to
return the current mounted directories. Also moving function to sys.c

Also fixes hashdump command to correctly print the warning if file is a
mounted directory.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-05-29 14:42:54 -07:00