Shellcheck provides many useful recommendations for writing shell code.
This commit makes testlib.bash shellcheck compliant.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When in a test we create a new version and a bundle update in that new
version, we copy the latest tar for each fullfile from the previous
versions into the new version. The code currently was searching for
those tars starting with the current directory, this was working fine
when running the tests locally because tests are run serially, but this
was causing an unexpected result when running in Travis since tests are
run in parallel, so this was causing the search to sometimes find the
tar in a different test environment (from another test), which was
causing unexpected results and a race condition.
This commit fixes the issue by narrowing the search of the tars to the
web-dir of the specific test environment.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Sometimes is useful to be able to limit the size of a test environment
to validate some scenarios that are bound to disk size.
This commit adds a -s (size) option that can be used when creating test
environments so the user can control how much space is available in that
environment. If not used, the test environment works the same as before.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When running verify --fix, if there are files in the system that are
marked as deleted in the manifest, then they are removed from the
system. However, if a component of the path of the file is a
symbolic link, the file is determined to be unsafe to be removed and
it is left in the system, the user is warned, and the not-deleted
file count is incremented.
This commit modifies this behavior so verify just skips the file, the
user is not warned with the "Not safe to delete: <FILE>" error message,
and so the not-deleted file count is not incremented since it is really
not useful for the end user.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The test library provided capabilities to create bundles that contained
symlinks to files in it, but it was not possible to create bundles with
symlinks that pointed to directories.
This commit adds the capability to create a test bundle that contains a
symlink to a directory by using the option "-c" when creating the
bundle.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When creating a test bundle that contains a symlink like this:
create_bundle -L -n my_bundle -f /my_file -l /my_link my_test_environment
A test bundle called my_bundle is created in my_test_environment,
this bundle has one link called my_link and this link points to a
file that is also added to the bundle's manifest. The problem is
that during the creation of the bundle the symbolic link is created
to point to an existing file but later in the process the file is
renamed, causing the symlink to become dangling. This dangling link
is then added to the bundle's manifest. If we run "swupd verify
--fix" the installed bundle will always show a hash mismatch and the
good symlink will be replaced with the dangling symlink.
This commit changes the way symlinks are being created when creating a
bundle so they point to the correct file.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Bash (test framework) uses the file descriptor 3 to print messages on
screen when running with the option "-t". The testlib provides a print
funtion that can be used to redirect the output to this file descriptor
easily. However if the print function is used and the testlib was
sourced and used from the command line instead if from a test script,
the shell will complain about an invalid file descriptor 3.
This commit fixes the issue by redirecting the message to the file
descriptor 3 only if it is available, if is not, it will use std out
instead.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When starting the web server from the test library, we wait for the
server to be up by doing some http requests and waiting for the server
to respond to them. When the server finally responds it sends some data
which pollutes the std output.
This commit silences those connection tests so we don't see any response
from the test server on the screen.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Testlib provides a function to create random files that are used
when creating test bundles. The size of these files vary but
it is always fairly small, which is best for most test scenarios. In
some special circumstances it would be helpful to have bundles with very
large files, or just with files of a known size.
This commit adds the ability to choose the file size when creating this
files via testlib.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The tests that attempt to verify the client certificate were using the
swupd -u option to set the url of the "upstream server". Recently a
function was added to the test library to set the default upstream
server.
This commit modifies these tests so they used the new function to set
the upstream server instead of using the -u option. This will provide
the benefit of the test environment being more similar to a prod
environment, also makes the test setup less convoluted and the tests
easier to read.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The server.py script is used in some functional tests to simulate
a slow content server. This commit provides a few extra options
that extend the use of this script for testing purposes.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When running functional tests using "bash -t" shows info about the test
being run. This commit adds some separators to the output so it is
easier to identify when the output of one test finishes and the next one
starts.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Every test environment gets created with a default value for versionurl
and contenturl that points to the appropriate place using the "file://"
protocol. In some occassions the user may want to change this value to
something different, for example if he/she is planning on using an http
server to host the version/content. This commit provides a couple of
functions that allow the user to change the values of the contenturl and
versionurl files.
This commit also makes a small improvement on the start_web_server
function which was sometimes causing the python server to hang.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When using the test generator from the testlib to create a new empty
test, it will pre-fill the user name and email from the git config in the
test if possible.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
There was a change of plans in the mixer side and looks like the flag
that will be used to mark a bundle as an experimental will be the 2nd
one instead of the 4th one.
This commit moves the flag to the 2nd position in the swupd client.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commits adds a function that creates a mirror (copy) of the
web-dir content and sets up the mirror in the target-dir as if the
user had used the swupd mirror --set command.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The versionurl and contenturl files are used by swupd to see where
the upstream server is.
This commit adds those files into the test environment and sets
them up so they point to the appropriate content in the test
environment, this way we can get rid of the -u option for most of
the tests, also this resembles closer a real environment.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
There was a bug defining the path for a test in the test generator
that was causing the script to fail if the command was executed from
the wrong path.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The check-update tests should use a group code of CHK, but the
test library was using AUT incorrectly which belongs to the autoupdate
tests.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Tests will be cleaned up by a trap routine when interrupted early. This
change also moves test web server and certificate store clean up to the
destroy_test_environment function.
Signed-off-by: John Akre <john.w.akre@intel.com>
Some tests wrote files and directories to /tmp. This change moves the
test certificates directory into the swupd repo's top level directory
and moves other test files into their corresponding test directories.
Fixes#650
Signed-off-by: John Akre <john.w.akre@intel.com>
Every test should include a unique ID, so it is necessary to have
a way of knowing what is the next available ID for the group of
tests.
This commit helps with that issue by implementing the following
features in the test library:
- Adds the function get_test_list which can be used to print a
list of tests in a specific group directory.
- Adds the function get_next_available_id which return the next
test ID that can be used in a new test.
- When using the generate_test function to generate an empty test
the next available ID will be used for the template.
This commit also adds the ID to a bundle-add test that was missing it.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When running tests, there is the possibility of preserving your
test environment for debugging purposes.
We were preserving the environment by skipping the execution of
the destroy_test_environment function when the DEBUG_TEST env
variable was set. This had one limitation, this was being done in
the default teardown function, so if the user would define his own
teardown then the DEBUG_TEST had no effect, the same if the user
used global_teardown instead. Another limitation the current
implementation had was that when an environment had been preserved
if the user would forget to clean it up manually before re-running
the test it would cause unexpected results in the test.
This commit removes both limitations by looking for the DEBUG_TEST
variable directly within the destroy_test_environment function so
the env will be preserved regardless of where the function is called
from and removes the environment at the beginning of the test if it
exists.
Closes#651
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When you do a "verify --fix --picky -m older_version", and one
or more of the currently installed bundles didn't exist back then,
a non useful error message was being used, and the verify operation
would fail.
This commit makes the following changes.
1) With this commit, if the mentioned situation happens, the verify
operation will still be stopped, but a useful message will now be
displayed to inform the user of the situation, and he/she will be
presented with an alternative to proceed using the --force option.
2) If the user uses the --force option, the user will still be warned
but the verify operation will continue and the offending bundles will
be removed from the system.
Closes#608
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
There were two web server implementations in the functional tests and the
test library's slow server implementation was not functional. These issues
were addressed by the following changes:
- Consolidate test web server implementations into test library
- Add partial download support to start_web_server test library function
- Fix test library slow server functionality.
Signed-off-by: John Akre <john.w.akre@intel.com>
This commit adds a few improvements to the test template that aim to
make easier to track tests.
As we are trying to organize the swupd tests better, it is useful to
have an easy way to identify who the original author of the test was
in case there are questions regarding the purpose of the test.
It is also useful sometimes to add a long description to a complex
test, so people reading the test understand what the test is attempting
to verify, and why is useful or different.
Lastly, it is a good practice to add a unique ID to every test case,
this makes it easier to track it, and to refer to it.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
I see the checkupdate/chk-update-slow-server test fail when the
http_proxy env variable is set, but no_proxy is not set.
Resolve the issue by adding "localhost" to no_proxy if needed, since the
test web servers run on localhost.
Signed-off-by: Patrick McCarty <patrick.mccarty@intel.com>
Some time ago the "-u" option of the create_test_environment and
create_version functions was renamed to "-r". Four of the verify
tests were not updated, so they were crashing during the setup,
while trying to create the test environment, so the test would not
be run since the setup failed but the teardown would still run and
the teardown would attempt to delete a non existent test environment
which would cause the destroy_test_environment to exit the script
causing an inconclusive test result.
This commit fixes the issue by doing the following actions:
- Fixes the tests by using the correct option "-r".
- Modifies the destroy_test_environment function so it does not exit
if a test environment doesn't exist, but instead it just returns a non
zero code and warns the user so the teardown works as expected.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When performing an update, if one or more files fail to be downloaded
the update process continues and the user is not notified of any
failure in the update.
This commit forces the update to abort if one or more files could not
be downloaded and informs the user about the failed update.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The function load_mom was returning the requested MoM if found or
NULL if not found.
This commit removes the load_mom_err function and adds the capability
of returning the error code to the load_mom function.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Linux programs return code should be a number between 0 and 255, so 404 is an
invalid return code. Removing it from docs and from swupd-error.h file.
As it wasn't being used, this won't be a problem.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This commit adds a menu of the functions that are available for
users when writing tests or just using the library to create
resources to test their code.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Soon, a minversion header field will be included in the MoM. This change
updates the test library to add a minversion header field to MoMs and adds
functions to create a minversion update. Also, the skip-verified-fullfiles
test was replaced by the update-minversion test. The update-minversion test
verifies that unchanged files with a version bump caused by a minversion
update are skipped during an update.
Signed-off-by: John Akre <john.w.akre@intel.com>
The test library contains a few assertions meant to verify the
existence or non existence of files and directories. In the case
of swupd, it is sometimes common that these files can be in directories
that are not readable by non root users, which is the case of the state
directory.
This commit makes a change so these assertions use root for testing
the existence of these files and directories so we don't get false
positives in the tests.
The commit also adds one more env variable with the --keepcache
option so we can use it in tests that require the state directory
to be persisted.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When a user wants to print to the terminal from a test that
is using the bats framework, it is necessary to use file descriptor
3.
This commit provides a wrapper function called print so it is
easier for user to print to terminal from tests.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When updating the manifest version we were replacing any line with the word
version to "version: new_version". This is a problem now that we are adding
a minversion field to the manifest. That could be a problem too when there
was a file named version.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This commit adds the iterative manifests (to-manifest) and the delta
manifest (from-manifest) when creating a bundle update. This manifests
should contain only those files that changed in the latest version.
Closes issue #557
There are some cases in the library where sed or awk is used to find
certain lines within files. To match these lines we use a pattern.
In some cases this pattern should be anchored to the beginning of the
line, in case a string with that pattern appears in a file (for example
in a bundle manifest name) in the last field. This is unlikely, but
it's better to avoid unintentional matches.
When a user creates a new version in a test environment and uses
the -r option an updated version of the os-release and format files
are added to the files directory. This is useful for some tests
like update tests.
This commit adds a second step of adding the updated files to the
os-core bundle so it is easier to create updates. With this commit
if using the option -r it means the new version will include an
update to the os-core bundle.
To support clients that need to authenticate with a web server
containing protected swupd content, an SSL certificate should be
presented if defined. This commit enables client certificate
authentication with swupd and adds relevant tests.
credit: gtkramer implemented client certificates in swupd
Signed-off-by: John Akre <john.w.akre@intel.com>
This commit adds the following test objects to testlib:
- start_web_server
- destroy_web_server
This change moves the implementation of web server creation and
destruction from individual tests to the test library.
Signed-off-by: John Akre <john.w.akre@intel.com>
This commit adds functions to testlib for the following:
- generate_certificate
- create_trusted_cacert
- destroy_trusted_cacert
These functions are used to create a self-signed public/private key
pair, create a trusted key store for the test environment with a public
key, and to delete the trusted key store. For the test environment to use
the trusted key store, swupd must be configured using
--with-fallback-capaths=<path to key store>. This configuration option
has been added to autogen and travis.
Signed-off-by: John Akre <john.w.akre@intel.com>
When "uninstalling" a bundle in web-dir it is better not to
remove all files in the bundle's manifest because they may be used
by another bundle, and checking every manifest would be too expensive
for this test task. Removing the bundle manifest and its packs is
enough for testing purposes.
Some functions from the test library can be used by end users to
modify some object from their test environment, for example using
the add_dependency_to_manifest they can add one or many dependencies
to a bundle's manifest. After modifying a bundle's manifest, many
subsequent actions need to occur in order for the objects to still be
valid, the manifest's tar needs to be re-created, since the manifest's
hash changed, the MoM needs to be updated with this new hash, etc.
To minimize the actions an end user needs to do, most of these
activities are automatically performed by most library functions.
However if the function is being called by another function which
expects to perform many actions in the same object, it may not be the
best option to be re-creating those tars and updating the MoM many times
for the same object since it could affect the performance.
This commit adds a -p flag to some functions to tell the functions you only
want a partial update, meaning that you plan on making more changes to the
test object so all those unnecessary changes are not performed.
Normal test environments come with a minimal version of os-core
bundle by default (only one file and the tracking file). Users
also have the option of not having os-core at all with the -e
"empty" option.
This commit adds another option "-r" for creating a test environment
that includes a more complete version of the os-core bundle,
with a couple of useful files: os-release and format. This type
of bundle can be useful when working with some type of tests like
updates.
Castulo Martinez