Currently sys_rm_recursive was used in any instance of deleting swupd
content from the system (update, repair and bundle-remove). This can
cause user data loss when unkown files are in directories that swupd
is deleting.
To prevent this, this patch changes how deleting content in swupd
operates. Swupd content removal is now done with sys_rm and the return
value is checked in case the removal failed due to a directory that
still had files in it. When this specific failure occurs, the
directory is added to a new list for reprocessing removals as it is
expected once the rest of the deletes on the system occur the failures
will go away as the directories will be empty (these deletes are
processed in alphabetical reverse order so leaf directories are
processed first). If the removal fails again it is presumed the
contents of the directory are not files swupd knows about and as such
should be kept somewhere else.
For handling the retention of user data, directories (with only the
content unknown to swupd) are renamed (currently using a
.deleted.$timestamp. prefix of the old name) and stored at the same
directory level they were previously found with one exception. The
exception is for nested deleted content best illustrated with an
example:
/swupd-dir1/user-file1
/swupd-dir1/swupd-dir2/user-file2
When swupd tries to remove the /swupd-dir1 content, it will store the
user files as follows:
/.deleted.$timestamp1.swupd-dir1/user-file1
/.deleted.$timestamp1.swupd-dir1/.deleted.$timestamp1.swupd-dir2/user-file2
To demarcate what was part of swupd content vs user content.
Signed-off-by: William Douglas <william.douglas@intel.com>
Instead of relying on a custom global_setup and global_teardown
functions, migrate to the use of bats' setup_file and teardown_file
functions.
Signed-off-by: William Douglas <william.douglas@intel.com>
Environment variables are used everywhere in testilb. This environment
variables are global variables that define the way testlib behaves.
However is was confusing to use the variables because they were
inconsistent between each other, for example some variables that define
paths would have absolute paths while other would have relative paths,
making it error prone while using them.
This commit makes the environment variables more consistent by following
a name convention for each type of variable, as an example, variables
that define absolute paths follow this convention ABS_<path_name>_DIR,
while variables that define relative paths are defined like this
<path_name>_DIR.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Current implementation was only removing test enviroment for envs created
on test setup. And test_teardown function needed to be overriden in case
global_setup() was used.
Now always cleaning the enviroment for both use cases, test_setup() and
global_setup(). The user doesn't have the control anymore on that and
if the user wants to keep the env KEEP_ENV variable should be used.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Test environment can be created on global_setup() or test_setup() and that
change the behavior of several operations on testlib.
Because of that, let the user create the test environment as desired instead
of forcing it on test_setup()
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This patch adds 2 scripts used to balance test execution. The weight_tests.bash
runs all tests and sets a weight to them based on how long they take to run. The
other, filter_bats_list.bash, use this information to split the tests in groups to
be executed by github actions.
When a new test is added the script will consider it with an average weight, so this
shouldn't unbalance the system right away. After some time, if we notice that the
system is not balanced anymore we can just run the weight_tests.bash again to rebalance.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
If the -R / --recursive flag is used, remove the specified bundles along
with their "deletable" dependencies.
In this context deletable means:
- the dependency is not os-core
- the dependency is not required by other installed bundle that is not
to be removed
- the dependency is not tracked (specifically installed by the user)
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit does a major refactor of the bundle-remove command to
include these improvements:
- Adds a "--force" flag that can be used to remove a bundle that is
required by other installed bundles along with all the bundles that
depend on it
- When attempting to remove a bundle which is required by other
installed bundles, swupd used to show the list of dependencies in a tree
view, this list was very large in some cases. This commit changes the
output of such cases from a tree view to a simple deduplicated list of
bundles that require the bundle to be removed. The user can still see
the old tree view by including the --verbose when running the command
- Stops notifying telemetry of user errors, like user trying to remove
an invalid bundle, or a bundle that was not installed, etc
- Consider all bundles provided in the bundle-remove command when
checking for dependencies (required by)
- Instead of removing each bundle in the command one by one, consolidate
all bundles to be removed, validate them and remove them all at once.
This will optimize the tasks to be executed for removing bundles.
One change included in this commit for bundle-remove is propagated to
bundle-list:
- When using "bundle-list -D BUNDLE" to find what bundles depend on
BUNDLE. The list will be presented in a simplified deduplicated list of
dependencies by default instead of the tree view that used to be
default. The user can still see the old tree view by running the command
appending the --verbose flag
Closes#891Closes#732Closes#674
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Renaming codes following these rules:
- All codes should start with SWUPD_ so we never get confused what
code comes from swupd
- Always separate codes that contain multiple words using an underscore
SWUPD_MULTI_WORD_ERROR instead of using SWUPDMULTIWORDERROR
- Code 0 is reserved for SWUPD_SUCCESS (or SWUPD_OK)
- Code 1 is reserved to be used as "no" by commands that return a boolean
state (e.g. swupd autoupdate, check-update)
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When running bundle-remove to remove a bundle the number in "Total
deleted files" is often wrong. This is happening because when deleting
the files from a bundle sometimes we start by deleting a directory
(with all the files within it), so all the files that got deleted are not
accounted for, only one file will be counted as deleted (the directory),
this will yield to an incorrect count of deleted files.
This commit fixes this issue by printing the total number of
files that should be deleted, minus those that failed to be deleted,
instead of counting those that succeeded.
Closes#690
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit adds the test ID to every bundle-remove test, removes
duplicates and make readability improvements.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
William Douglas