Instead of initializing curl library on start, initialize it only when needed.
The advantage of this approach is that we can run some commands offline, if
there's no file to download.
Fixes#801Fixes#895Fixes#277
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When running the check-update command we get the latest version for the
format we are currently in, but we really want to get the latest version
regardless of the format.
This commit fixes the issue.
Closes#482
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Estimating the total download size is almost as slow as downloading the
manifests. So just don't do it.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
If --update-search-file-index flag is used on update, all search-file indexes, i.e.
all Manifests will be downloaded on update. This shouldn't be used if you have
disk or network restrictions, but it shouldn't be very download intensive after the
first usage because delta manifests are going to be used.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Previously, when adding included manifests errors were thrown away which
resulted in swupd operations that could succeed, even when failing to
load an included manifest. Now when an included manifest fails to load,
swupd will return a failure.
Signed-off-by: John Akre <john.w.akre@intel.com>
When using testlib to create tests, ignore lists can be used which
include lines of output that should be ignored when running tests.
The ignore lists can exist at three different levels:
- an ignore file that applies to all tests
- an ignore file that applies to all files of one theme
- an ignore file that applies to all tests within the same file
There was a bug in this last one that was causing ignore-list files to
be unrecognized unless the test number was also used as part of the
ignore-list name. This was causing issues because the test number
can change depending on how you run the test.
This commit fixes the issue.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Circular includes shoudn't exist, but we can handle that without crashing.
Also-add circular includes are common and expected. So we just add the
first occurence found.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
In the case that we have a bundle installed as also-add and removed later,
repair shouldn't reinstall that bundle. The same is valid for diagnose and update.
Os-install is the only exception. We should always install all bundles that are
listed as also-add in os-install.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When doing an update, we check to see if a mirror is stale, this only
needs to be done if there is a mirror set and if the upstream server is
up and reachable.
This commit checks to see if a mirror is set and there is an upstream
server to compare against, and only then it checks to see if it is stale.
Closes#922
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Currently users can set content and version urls based on http or https
protocols. This pose a security risk if users decide to use http.
This commit blocks swupd from working with http unless is specifically
allowed by using the --allow-insecure-http flag.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit adds a few more cosmetic changes to the commands that run
verify in the back for consistency.
- Different steps in the update process are separated by a blank line.
- Messages from swupd should not finish with a '.'
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The update/update-json.bats functional test is giving a false positive
since one hardcoded value (time) was taking longer to run, than in a
local environment.
This commit fixes the issue by using a regex to accept any time value.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Adding messages where swupd could take some time to finish a step so
users know better where the process is at.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The json tests had been disabled because they were unstable. This issue
was fixed in a previous commit, so these tests should be enabled again.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit makes use of the swupd_progress_callback() function to
report progress downloading fullfiles based on how much data we have
downloaded vs the number of files downloaded. This callback will only be
used when the number of files to be downloaded are less than MAX_FILES,
calculating the total download size can be very costly if the files are
too many. If the files to be downloaded are more than MAX_FILES we will
fallback to reporting download progress based on file count as before.
When installing bundles or doing updates, swupd creates a list of files
that need to be downloaded. This list may contain files that were
already downloaded via packages and it often does. These files are then
skipped at the moment of downloading them since they are already in the
system. This causes a misleading output that shows the user that
fullfiles will be downloaded when they are actually not.
This commit filters the list of fullfiles to be downloaded to only
contain those ones that actually need to be downloaded.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit makes use of the swupd_progress_callback() function to
report progress downloading packs based on how much data we have
downloaded vs the number of files downloaded.
This commit also fixes a bug in the download_subscribed_packs function.
Swupd was not downloading the correct pack for bundles not installed in
the system that had been recently added as dependency of another
installed bundle.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Disabling json tests that checks for progress because tests are unstable.
If we have any minor changes in the code that could affect how curl is called
we could have different progress reports and because of that we would have false
negatives.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Note that we don't need to use run_command() (fork + exec) because
we don't need to handle any output on swupd. We can just replace current
process with the new swupd execution.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The --json-output flag can be enabled for every swupd command.
This commit converts it to be a global flag, and enables it in every
swupd command.
Closes#869
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
swupd prints some messages to stdout and others to stderr, this makes
the code somewhat confusing and also it creates a mess regarding which
messages go where. Messages are usually printed using fprintf() and
printf().
This commit replaces all calls to printf and fprintf with one of the
functions that are part of the logger. This will bring many advantages
to the code:
- readability, it is straightforward to identify where a message needs
to go based on the name of the logger functions.
- consistency, the output will go to either stderr or stdout depending
on the type of message being printed.
- the format of the messages will also be consistent, so for example,
if the message is an error the message will automatically include the
"Error: " label in the beginning of the message, this way we avoid
having things like "ERROR:", "error - ", etc...
Closes#864Closes#294Closes#489Closes#433
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When using bundle-add to install bundles with the --json-output flag,
the output generated by swupd will be formatted to json. This is useful
for other applications that are reading the swupd output to determine
real time progress.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When creating an update that updates a file, the file delta is added to the
delta pack to update from the previous version, but the delta should be in
fact in all the delta packs previous to that one as well so a user could
use a delta pack to update from any previous version to the latest one.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When creating an update that adds a file, the file is added to the delta
pack to update from the previous version, but the new file should be in
fact in all the delta packs previous to that one as well so a user could
use a delta pack to update from any previous version to the latest one.
This commit adds that new file to all delta packs from versions older
than the one containing the update.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When running an update using the flag --time/-t the user is presented
with a verbose time output of the update operations. The time collected
for those operations was wrong.
This commit fixes the issue and presents a verbose output that is
consistent with other commands like bundle-add --time.
Closes#584
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The update_minversion and bump_format testlib functions both perform a
minversion update, but with separate implementations.
This commit consolidate both implementations.
Closes#618
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The bump_format testlib function created new minversions for the +10 and
+20 versions, but it should only create a new minversion for the +20
version. This change copies the +20 update content into the +10 update
and preserves the versions of unchanged files in the +10 version. Also,
this change updates the minversion header field for the +20 version to
reflect a minversion update.
Signed-off-by: John Akre <john.w.akre@intel.com>
Instead of printing errors and warnings, just print debug information
when we are looking for the right ca path to use. If we couldn't
connect to the server, abort and print a clear error message.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Before downloading full manifests try to use manifest deltas if available.
If deltas are missing, use full manifest.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Use similar output and same exit code used when you are already in the
last version in the case where you are updating to the same version you
already are.
i.e. if you are in version 100, both commands are going to have the same
exit code:
- swupd update
- swupd update -m 100
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The retry code in swupd is not an ideal implementation, making it
difficult to use throughout the code base. Some parts retry
differently based on needing to support retries for network errors,
or retry based on IO errors.
This commit refactors this code into a consistent implementation so
the retry code is not spread across the code base.
Closes#211
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When swupd needs to run a boot script it does so without checking if the
file exists in the system or not. This will cause an error to be raised
by the shell if the script does not exist.
This commit adds a check for the existance of the script before
attempting to run it.
Closes#795
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Regardless of what codes are used internally, swupd should always
exit with a code defined in swupd_exit_codes.h.
This commit makes those changes for update.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Renaming codes following these rules:
- All codes should start with SWUPD_ so we never get confused what
code comes from swupd
- Always separate codes that contain multiple words using an underscore
SWUPD_MULTI_WORD_ERROR instead of using SWUPDMULTIWORDERROR
- Code 0 is reserved for SWUPD_SUCCESS (or SWUPD_OK)
- Code 1 is reserved to be used as "no" by commands that return a boolean
state (e.g. swupd autoupdate, check-update)
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This will make outputs a lot better when using --quiet because optional
curl messages wont be printed.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The biggest change is handling timeout in a better way. The response
code in a timeout (and some other curl errors) is the response from
the last successful operation, so on timeouts we can get 200 (OK)
response code instead of 206(partial content) in a partial download. So
we should look at curl return and not on http return code to decide if we
are going to try to resume the download or not.
Changing slow-server test because error message has changed
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Group in a function the checks for http response code and curl return,
so we can use it for parallel downloads.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Two PRs that included tests for swupd update were submitted and merged
at the same time, which caused the test IDs to be inconsistent.
This commit fixes the IDs of the update tests with duplicated ID.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit adds a few tests that validate the behavior of swupd when it
tries to update a system that has a full disk in different stages of the
update.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
We were using an http server based on the SimpleHTTPRequestHandler to
run most of our server-related tests except for one which was using an
http server based on the BaseHTTPRequestHandler.
This commit merges the functionality of both server clasesses into one
so we use the same one for all future tests.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
As part of the swupd initialization, curl is initialized, which means it
is configured and the connection is tested. When configuring curl for
this test, no timeout was being configured, which was causing curl to
hang waiting for a response from the server if the server was unresponsive.
This commit configures the curl timeouts the same way they are being
configured when downloading files, so in case we get an unresponsive
server (or mirror) it doesn't get stuck forever.
Closes#669
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When we have SSL connections errors we try to use fallback CA paths added by the
--with-fallback-ca-paths configuration flag.
Instead of printing errors in this case, just print a warning. An error
message will be printed when we fail when trying all fallbacks.
Fixes: #746
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
The tests that attempt to verify the client certificate were using the
swupd -u option to set the url of the "upstream server". Recently a
function was added to the test library to set the default upstream
server.
This commit modifies these tests so they used the new function to set
the upstream server instead of using the -u option. This will provide
the benefit of the test environment being more similar to a prod
environment, also makes the test setup less convoluted and the tests
easier to read.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The slow server tests are using the global setup to create the test
dependencies, like setting up the web server. Recently a function to set
the upstream server in a test environment was added to the test library
which now makes possible to configure the environment to use this server
after it's been created so we no longer need to create the web server in
the global setup.
This commit moves the creation of the web server to the test_setup
function to make the code from the tests easier to read and less
convoluted.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Otavio Pontes