The current format is very useful information to get from bug reports
so add it as default output to swupd update.
Signed-off-by: William Douglas <william.douglas@intel.com>
When a bundle is removed, ensure the new orphans created are detected
by bundle-list and bundle-remove.
Signed-off-by: William Douglas <william.douglas@intel.com>
Instead of relying on a custom global_setup and global_teardown
functions, migrate to the use of bats' setup_file and teardown_file
functions.
Signed-off-by: William Douglas <william.douglas@intel.com>
Environment variables are used everywhere in testilb. This environment
variables are global variables that define the way testlib behaves.
However is was confusing to use the variables because they were
inconsistent between each other, for example some variables that define
paths would have absolute paths while other would have relative paths,
making it error prone while using them.
This commit makes the environment variables more consistent by following
a name convention for each type of variable, as an example, variables
that define absolute paths follow this convention ABS_<path_name>_DIR,
while variables that define relative paths are defined like this
<path_name>_DIR.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Currently there is only one location, known as "statedir", where swupd
stores data and cache. Normally it make sense to keep the data in a
path that is relative to the path prefix of the target system, since the
data is specific to that system, but cache can be independent of the
system, and that way it can be reused for other target systems if
desired.
This commit splits the cache and data into two different locations.
It also stores the cache in a path that is dependent of the mirror url
being used by swupd. The commit also makes all references to files in
the cache or data directories through getter functions thus removing the
hardcoded reference to the file's location, making it easier to move the
location in the future if needed. Lastly, this commit changes the read
permissions for the data directory and the manifest directories in the
cache to be user readable.
When swupd downloads manifests to the statedir they are downloaded in
directories that represent the version of clear that the manifests
belong to. However those version directories are stored in the statedir,
root directory. This commit moves them into their own "manifest"
directory so there is more consistency with the downloaded files.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When swupd downloads manifests to the statedir they are downloaded in
directories that represent the version of clear that the manifests
belong to. However those version directories are stored in the statedir,
root directory. This commit moves them into their own "manifest"
directory so there is more consistency with the downloaded files.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Adding the --orphans flag to bundle-list which shows those bundles that
are installed ut no longer required by any tracked bundle.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Use global_setup in all bundle-list tests so we can only generate the
test environment once and use it for multiple tests. The one exception
is if we need to modify the env for a specific test.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit provides a minimal output to be displayed when the --quiet
flag is used for these commands:
- swupd bundle-list
- swupd 3rd-party bundle-list
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When listing bundles using "swupd bundle-list --status" we were
distinguishing bundles that were explicitly installed, by marking them
as such and since bundle-list shows only bundles tha are installed, bundles
that were not marked as explicitelly installed were assumed to be
just installed.
When we use "bundle-list --all --status" (which shows installed and
uninstalled bundles), we were marking files as "installed", "explicitly
installed" and bundles without any of these distinctions were assumed to
be not installed. This could cause confusion.
This PR changes the "bundle-list --status" so we always label bundles
as either "implicitly installed" or "installed" (even though it couls
sound a little redundant) so it is consistent with the way they are
labeled during "bundle-list --all --status".
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When creating test bundles, we are not including os-core in them by
default. This is how it works in production so we should do it so it
resembles the real production environment more accurately.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Current implementation was only removing test enviroment for envs created
on test setup. And test_teardown function needed to be overriden in case
global_setup() was used.
Now always cleaning the enviroment for both use cases, test_setup() and
global_setup(). The user doesn't have the control anymore on that and
if the user wants to keep the env KEEP_ENV variable should be used.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Test environment can be created on global_setup() or test_setup() and that
change the behavior of several operations on testlib.
Because of that, let the user create the test environment as desired instead
of forcing it on test_setup()
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2 groups were created:
- slow: Tests that are very slow because of sleeps, so we can run them all
in parallel
- system: Tests that make change to the system, so they can cause problems
when running in parallel, so we run them in series.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This patch adds 2 scripts used to balance test execution. The weight_tests.bash
runs all tests and sets a weight to them based on how long they take to run. The
other, filter_bats_list.bash, use this information to split the tests in groups to
be executed by github actions.
When a new test is added the script will consider it with an average weight, so this
shouldn't unbalance the system right away. After some time, if we notice that the
system is not balanced anymore we can just run the weight_tests.bash again to rebalance.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The "bundle-list" command can be used to show all installed bundles in
the system, however, currently there is no way to know which of those
installed bundles were implicitly or explicitly installed.
This commit implements the "--status" flag for bundle-list which can be
used to show which bundles were explicitly or implicitly installed.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Sometimes is useful to see the list of dependencies in a tree view form,
for example when trying to remove bundles from a system.
This commit adds the ability of listing the dependencies of a bundle in
a tree view when --verbose is used.
Closes#929
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
For some options of bundle-list, like --deps and --has-dep, we sometimes
need to download a considerable amount of manifests, if that happens
swupd just seems to hang for some time, this time could be considerably
in slow networks.
This commit enables the spinner while the manifests are baing downloaded
during a bundle-list operation so users know swupd is not hung and it is
doing some downloading.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When listing bundles using the --deps or --has-dep flags which
take a BUNDLE as argument, one or more repositories may not have
the specified BUNDLE, but we should not return a
SWUPD_INVALID_BUNDLE error unless the bundle is not found in any
repository.
This commit enforces that behavior and also modify the output of the
different list options to make them more consistent with one another.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit does two things:
- makes the output of all options of bundle-list consistent with each
other.
- makes the ouptut of all bundle-list work properly with the --quiet
flag so it prints the output in a format that is friendly for piping the
output.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit does a major refactor of the bundle-remove command to
include these improvements:
- Adds a "--force" flag that can be used to remove a bundle that is
required by other installed bundles along with all the bundles that
depend on it
- When attempting to remove a bundle which is required by other
installed bundles, swupd used to show the list of dependencies in a tree
view, this list was very large in some cases. This commit changes the
output of such cases from a tree view to a simple deduplicated list of
bundles that require the bundle to be removed. The user can still see
the old tree view by including the --verbose when running the command
- Stops notifying telemetry of user errors, like user trying to remove
an invalid bundle, or a bundle that was not installed, etc
- Consider all bundles provided in the bundle-remove command when
checking for dependencies (required by)
- Instead of removing each bundle in the command one by one, consolidate
all bundles to be removed, validate them and remove them all at once.
This will optimize the tasks to be executed for removing bundles.
One change included in this commit for bundle-remove is propagated to
bundle-list:
- When using "bundle-list -D BUNDLE" to find what bundles depend on
BUNDLE. The list will be presented in a simplified deduplicated list of
dependencies by default instead of the tree view that used to be
default. The user can still see the old tree view by running the command
appending the --verbose flag
Closes#891Closes#732Closes#674
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
There was a bug in the logic of the setup function in testlib which was
causing the globale_teardown to be incorrectly called when tests were
being run using "bats <directory>/". This commit fixs the issue.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Instead of initializing curl library on start, initialize it only when needed.
The advantage of this approach is that we can run some commands offline, if
there's no file to download.
Fixes#801Fixes#895Fixes#277
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The --json-output flag can be enabled for every swupd command.
This commit converts it to be a global flag, and enables it in every
swupd command.
Closes#869
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
swupd prints some messages to stdout and others to stderr, this makes
the code somewhat confusing and also it creates a mess regarding which
messages go where. Messages are usually printed using fprintf() and
printf().
This commit replaces all calls to printf and fprintf with one of the
functions that are part of the logger. This will bring many advantages
to the code:
- readability, it is straightforward to identify where a message needs
to go based on the name of the logger functions.
- consistency, the output will go to either stderr or stdout depending
on the type of message being printed.
- the format of the messages will also be consistent, so for example,
if the message is an error the message will automatically include the
"Error: " label in the beginning of the message, this way we avoid
having things like "ERROR:", "error - ", etc...
Closes#864Closes#294Closes#489Closes#433
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Instead of printing errors and warnings, just print debug information
when we are looking for the right ca path to use. If we couldn't
connect to the server, abort and print a clear error message.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Regardless of what codes are used internally, swupd should always
exit with a code defined in swupd_exit_codes.h.
This commit makes those changes for bundle-list.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Renaming codes following these rules:
- All codes should start with SWUPD_ so we never get confused what
code comes from swupd
- Always separate codes that contain multiple words using an underscore
SWUPD_MULTI_WORD_ERROR instead of using SWUPDMULTIWORDERROR
- Code 0 is reserved for SWUPD_SUCCESS (or SWUPD_OK)
- Code 1 is reserved to be used as "no" by commands that return a boolean
state (e.g. swupd autoupdate, check-update)
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This will make outputs a lot better when using --quiet because optional
curl messages wont be printed.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The biggest change is handling timeout in a better way. The response
code in a timeout (and some other curl errors) is the response from
the last successful operation, so on timeouts we can get 200 (OK)
response code instead of 206(partial content) in a partial download. So
we should look at curl return and not on http return code to decide if we
are going to try to resume the download or not.
Changing slow-server test because error message has changed
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When listing bundle, if downloading the files fail because of disk
space, swupd should not re-attempt to download the files since it will
fail again for sure.
This commit adds tests that verify this behavior.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When we have SSL connections errors we try to use fallback CA paths added by the
--with-fallback-ca-paths configuration flag.
Instead of printing errors in this case, just print a warning. An error
message will be printed when we fail when trying all fallbacks.
Fixes: #746
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The tests that attempt to verify the client certificate were using the
swupd -u option to set the url of the "upstream server". Recently a
function was added to the test library to set the default upstream
server.
This commit modifies these tests so they used the new function to set
the upstream server instead of using the -u option. This will provide
the benefit of the test environment being more similar to a prod
environment, also makes the test setup less convoluted and the tests
easier to read.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit adds the ability to show an identifier in bundles
that are marked as experimental when listing installed bundles
(swupd bundle-list).
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit adds the ability to show an identifier in bundles that
are marked as experimental when listing available bundles (swupd
bundle-list --all).
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Mixer will soon start using a new modifier for bundles in a MoM
that are marked as experimental. These bundles will include a
"e" in the 4th position of the modifiers of a manifest, like shown
here:
M..e 4b91c3122e7e32f1e3edb597c6f89ff32cfffd977afb5e5 10 some-bundle
This commit adds the mechanism to parse that element from the MoM
and store it in memory. In the future this will allow swupd to treat
this experimental bundles differently if required.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Adding an ID to all bundle-list tests and rewording some of the
test descriptions to make it more readable.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Tests will be cleaned up by a trap routine when interrupted early. This
change also moves test web server and certificate store clean up to the
destroy_test_environment function.
Signed-off-by: John Akre <john.w.akre@intel.com>
Some tests wrote files and directories to /tmp. This change moves the
test certificates directory into the swupd repo's top level directory
and moves other test files into their corresponding test directories.
Fixes#650
Signed-off-by: John Akre <john.w.akre@intel.com>
The old bundlelist tests used the swupdlib.bash library that was
replaced by testlib.bash.
This commit replaces those old bundlelist tests with new versions
of the same tests that now use testlib.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The operations have different command result output styles. Also,
in particular bundle-remove error is a bit redundant.
This commit changes the messages displayed by the bundle-add and
bundle-remove functions so they have consistent messages for their
corresponding operations.
Fixes#475
William Douglas