Automated tests should create its own test environment including
all needed dependencies when the test starts, and should clean up
after itself once the test is over. Sometimes when developing
these tests, they don't work as expected at first and is difficult
to figure out what the problem is if the test environment is gone.
This commit allow users to temporarily skip the removal of the test
environment so he/she can debug what the problem was by only running
the test with the DEBUG_TEST env variable set to "true".
Example: DEBUG_TEST=true bats my_test.bats
This commit adds the following two assertions:
- assert_files_equal
- assert_files_not_equal
These can be used in tests for validating file equality consistently.
Some variables in the test library were missing the $ character,
whichs makes bash interpret them as strings instead of variables.
This commit fixes the variable names.
The test library should be used to create test cases based on
modular "steps", the test library helps you build these steps,
sometimes is difficult to see what parameters a function from
testlib needs.
This commit adds usage info for every function in the test lib so
it is easier for the person writing test cases to know what parameters
the function needs, by simply executing the function name with no
parameters in the command line will show this info.
Most of the functions of the test library include some type of
parameter validation, when a function is called and one of its
parameters don't comply with the expected parameters it terminates
the whole script. This is expected, but sometimes it can be
difficult to figure out what was the function that received the
incorrect parameters in the script.
This commit adds a function that prints a stack of the function
calls so when a script is terminated it is easier to figure out
the culprit.
The test library includes many assertions intended to validate
a command's output. These assertions take the command output and
compare it against an expected output. Some of these assertions
take a literal string and others take a string with regular
expression values. However, regardless of what king of string is
being used for the expected output, there are some common lines
in the output that are usually not important and are safe to ignore.
An example would be empty lines or lines with only dots.
This commit adds the capability for those assertions to use an
ignore file to include patterns that could be ignored/removed from
the command's output before comparing it against the expected
output. Also with this commit empty lines and lines with only dots
will be ignored by default. If the assertion needs to validate an
exact output without ignoring anything in the ignore-list nor ignoring
dots and blank lines, then the --identical option can be used with
the assertion.
This commit include two parts.
1) It adds 6 new assertions to the test library:
- assert_is_output
- assert_is_not_output
- assert_regex_is_output
- assert_regex_is_not_output
- assert_regex_in_output
- assert_regex_not_in_output
2) It migrates the tests from the following categories to use the
new library:
- completion
- hashdump
- mirror
- search
Finally this commit changes a few test cases so they use
"assert_is_output" instead of "assert_in_output", that way
we can have a tighter control of the output we want since
assert_is_output requires the whole output to match to a
specific text instead of doing a partial match.
This commit adds the following three funtions to the test library:
- install_bundle
- remove_bundle
- clean_state_dir
These functions are useful specially when the global_setup and the
global_teardown are used instead of test_setup and test_teardown.
These functions allow the user to create all of its test resources
in the global_setup and reuse those same resources throughout all
the tests in the test script. By using the global_setup and
global_teardown functions the test execution time can be greatelly
reduced.
This commit also modifies the add-rc.bats and remove-rc.bats to
leverage the global setup and teardowns to increase the execution
speed. These times were reduced as shown:
- bundleadd_v2/add-rc.bats from 1m50.129s -> 0m20.026s
- bundleremove_v2/remove-rc.bats from 0m59.079s -> 0m15.446s
A new test library.bash was introduced that simplifies the creation
of test environments and test objects.
This commit rewrites all existing tests from
test/functional/bundleremove and test/functional/checkupdate but this
time using the new test library.
NOTE: Once all the tests have been rewriten to use the new lib, they
can be included in the Makefile so they are run with every patch,
and the old tests can be deleted.
A new test library.bash was introduced that simplifies the creation
of test environments and test objects.
This commit rewrites all existing tests from test/functional/bundlelist
but this time using the new test library. Once all the tests have been
rewriten to use the new lib, they can be included in the Makefile so
they are run with every patch, and the old tests can be deleted.
This commit adds the last rewritten tests for bundle-add, these
tests are the equivalent to the ones in the bundleadd directory
but using the new test library.
This commit adds some assertions that can be used to validate
common things in a consistent manner among tests.
The assertions added with this commit are the following:
- assert_status_is
- assert_status_is_not
- assert_dir_exists
- assert_dir_not_exists
- assert_file_exists
- assert_file_not_exists
- assert_in_output
- assert_not_in_output
More assertions may be added as needed while migrating the rest of
the tests to use the new testlib.
There is some stuff that is common for every test file, so this
commit adds a function that generates a new test file based on a
template to facilitate test creation.
The old way of creating new tests require a lot of manual steps to
create the test resources needed to run the tests. This commit is
part of a series of commits to improve the workflow for creation of
new tests.
This commit introduces a new test library that aims to replace the
old swupdlib.bash library which provides many funtions to create
new test resources easily.
The following functions were added in this commit:
- generate_random_content
- generate_random_name
- validate_path
- validate_item
- validate_param
- create_dir
- create_file
- create_link
- create_tar
- create_manifest
- add_to_manifest
- create_test_environment
- destroy_test_environment
- create_bundle
- set_env_variables
- sign_manifest
The tests in bundleadd_v2 would eventually replace those in bundleadd
Related to issue #258
Castulo Martinez