This commit modifies some of the functions of the test library so it is
possible to update bundles that belong to 3rd-party repositories.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit separates the 3rd-party tests in their own job since we will
be adding several tests related to 3rd-party in the upcoming days.
This commit also rebalances the tests in the update groups so they are
better balanced.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
sudo swupd 3rd-party add <repo_name> <upstream_url>
With some clean up from Otavio
Signed-off-by: Karthik Prabhu Vinod <karthik.prabhu.vinod@intel.com>
When running tests, there is the possibility of preserving your
test environment for debugging purposes.
We were preserving the environment by skipping the execution of
the destroy_test_environment function when the DEBUG_TEST env
variable was set. This had one limitation, this was being done in
the default teardown function, so if the user would define his own
teardown then the DEBUG_TEST had no effect, the same if the user
used global_teardown instead. Another limitation the current
implementation had was that when an environment had been preserved
if the user would forget to clean it up manually before re-running
the test it would cause unexpected results in the test.
This commit removes both limitations by looking for the DEBUG_TEST
variable directly within the destroy_test_environment function so
the env will be preserved regardless of where the function is called
from and removes the environment at the beginning of the test if it
exists.
Closes#651
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Some functionality was added to the test library to allow printing
function availability plus usage information.
This commit adds that info to the readme of the library to keep it
accurate. It also adds a table of groups for the tests which will
be useful soon when we start using IDs in the tests.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Normal test environments come with a minimal version of os-core
bundle by default (only one file and the tracking file). Users
also have the option of not having os-core at all with the -e
"empty" option.
This commit adds another option "-r" for creating a test environment
that includes a more complete version of the os-core bundle,
with a couple of useful files: os-release and format. This type
of bundle can be useful when working with some type of tests like
updates.
Some tests need to have not only one but two or more versions of
the server side content (web-dir).
This commit adds a function that can be used to create these new
versions.
Tests that compare output with an expected output strip text in
ignore files first.
This commit adds a global ignore-list intended to be used as a
fallback for all tests.
When creating a bundle using the testlib functions, if the user
wants the bundle to include files, those files are generated
automatically with random content.
This commit adds the capability of specifying an existing file to
be used for the bundle by using the
"-f <file in manifest>:<file to use>" notation.
Automated tests should create its own test environment including
all needed dependencies when the test starts, and should clean up
after itself once the test is over. Sometimes when developing
these tests, they don't work as expected at first and is difficult
to figure out what the problem is if the test environment is gone.
This commit allow users to temporarily skip the removal of the test
environment so he/she can debug what the problem was by only running
the test with the DEBUG_TEST env variable set to "true".
Example: DEBUG_TEST=true bats my_test.bats
The test library includes many assertions intended to validate
a command's output. These assertions take the command output and
compare it against an expected output. Some of these assertions
take a literal string and others take a string with regular
expression values. However, regardless of what king of string is
being used for the expected output, there are some common lines
in the output that are usually not important and are safe to ignore.
An example would be empty lines or lines with only dots.
This commit adds the capability for those assertions to use an
ignore file to include patterns that could be ignored/removed from
the command's output before comparing it against the expected
output. Also with this commit empty lines and lines with only dots
will be ignored by default. If the assertion needs to validate an
exact output without ignoring anything in the ignore-list nor ignoring
dots and blank lines, then the --identical option can be used with
the assertion.
This commit include two parts.
1) It adds 6 new assertions to the test library:
- assert_is_output
- assert_is_not_output
- assert_regex_is_output
- assert_regex_is_not_output
- assert_regex_in_output
- assert_regex_not_in_output
2) It migrates the tests from the following categories to use the
new library:
- completion
- hashdump
- mirror
- search
Finally this commit changes a few test cases so they use
"assert_is_output" instead of "assert_in_output", that way
we can have a tighter control of the output we want since
assert_is_output requires the whole output to match to a
specific text instead of doing a partial match.
A new test library.bash was introduced that simplifies the creation
of test environments and test objects.
This commit rewrites all existing tests from
test/functional/bundleremove and test/functional/checkupdate but this
time using the new test library.
NOTE: Once all the tests have been rewriten to use the new lib, they
can be included in the Makefile so they are run with every patch,
and the old tests can be deleted.
This commit adds the last rewritten tests for bundle-add, these
tests are the equivalent to the ones in the bundleadd directory
but using the new test library.
Castulo Martinez