Previously, these messages were printed to STDOUT:
Overriding version and content URLs with...
Overriding content URL with...
Overriding version URL with...
But especially with --quiet, the output to STDOUT should be strictly the
requested data. So print these as warnings instead.
There are some problems with how we are using multiplexed curl causing
huge slowdowns (2-5 times slower).
For now rather than rework the multiplexed curl code, switch to using
the synchronous code path.
Signed-off-by: William Douglas <william.douglas@intel.com>
Environment variables are used everywhere in testilb. This environment
variables are global variables that define the way testlib behaves.
However is was confusing to use the variables because they were
inconsistent between each other, for example some variables that define
paths would have absolute paths while other would have relative paths,
making it error prone while using them.
This commit makes the environment variables more consistent by following
a name convention for each type of variable, as an example, variables
that define absolute paths follow this convention ABS_<path_name>_DIR,
while variables that define relative paths are defined like this
<path_name>_DIR.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Currently there is only one location, known as "statedir", where swupd
stores data and cache. Normally it make sense to keep the data in a
path that is relative to the path prefix of the target system, since the
data is specific to that system, but cache can be independent of the
system, and that way it can be reused for other target systems if
desired.
This commit splits the cache and data into two different locations.
It also stores the cache in a path that is dependent of the mirror url
being used by swupd. The commit also makes all references to files in
the cache or data directories through getter functions thus removing the
hardcoded reference to the file's location, making it easier to move the
location in the future if needed. Lastly, this commit changes the read
permissions for the data directory and the manifest directories in the
cache to be user readable.
When swupd downloads manifests to the statedir they are downloaded in
directories that represent the version of clear that the manifests
belong to. However those version directories are stored in the statedir,
root directory. This commit moves them into their own "manifest"
directory so there is more consistency with the downloaded files.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When swupd downloads manifests to the statedir they are downloaded in
directories that represent the version of clear that the manifests
belong to. However those version directories are stored in the statedir,
root directory. This commit moves them into their own "manifest"
directory so there is more consistency with the downloaded files.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Don't run heuristics for os-install or repair --quick.
On os-install we want all files to be always installed. On repair --quick
we don't process any do_not_update files
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This commit removes the extra slash '/' from the path from:
- the update_boot function
- the verifytime script
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Alternative solution to solution proposed on #1318
Instead of trying to create a link just don't try to link symlinks.
Tested patch using bsdtar and implemented automated testing to
validate solution.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When doing a system install with "os-install --bundles" if the list of
bundles to install includes only "os-core" the install fails.
This commit fixes the issue.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This patch adds 2 scripts used to balance test execution. The weight_tests.bash
runs all tests and sets a weight to them based on how long they take to run. The
other, filter_bats_list.bash, use this information to split the tests in groups to
be executed by github actions.
When a new test is added the script will consider it with an average weight, so this
shouldn't unbalance the system right away. After some time, if we notice that the
system is not balanced anymore we can just run the weight_tests.bash again to rebalance.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When using "bundle-info BUNDLE --dependencies", information regarding
BUNDLE is presented to the user, this information includes the list of
direct and indirect dependencies BUNDLE has. All direct includes that
are optional (also-add) are marked as such, while there is no way to
distiguish what indirect dependencies are optional with the provided
information.
This commit expands the information shown by the command so indirect
dependencies that are optional are shown as such along with info about
the status of that dependency.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
A spinner was added to swupd to show some kind of progress while
downloading content which its amount is unknown. Once the content is
downloaded the stopped spinner should be removed from the screen. This
commit does that.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit adds a function that can run an operation in one or all
3rd-party repositories. If a repository is specified then the operation
is run in only that 3rd-party repo, otherwise it is run in all 3rd-party
repos.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
When os-install is used to install an OS and the --bundles option is used
to specify some bundles that should be installed in the system along
with the OS, the bundles specified should be tracked.
Closes#1153
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Print a "Validate downloaded files" message to separate 2 different progress bars
and improve the output when downloading extra fullfiles
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Changing the progress report lib to always print 0 or -1 when a step is
started and a 100 when a step finishes.
Also reorganizing all commands to have less steps (when possible) and that
are more meaningful to progress report. Hide some internal steps that doesn't
provide useful information to end users.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This ensures we have a callback function
for full_file curl download. The function
implements a spinner for progress
Fixes#1060
Signed-off-by: Karthik Prabhu Vinod <karthik.prabhu.vinod@intel.com>
Never use on tests fake url without specifying the protocol because proxies can
respond differently. Use https pointing to localhost in this cases.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
Instead of initializing curl library on start, initialize it only when needed.
The advantage of this approach is that we can run some commands offline, if
there's no file to download.
Fixes#801Fixes#895Fixes#277
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
When the --download flag is set for the os-install command, update
content will be downloaded to the statedir and no content will be
installed.
Signed-off-by: John Akre <john.w.akre@intel.com>
Circular includes shoudn't exist, but we can handle that without crashing.
Also-add circular includes are common and expected. So we just add the
first occurence found.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
In the case that we have a bundle installed as also-add and removed later,
repair shouldn't reinstall that bundle. The same is valid for diagnose and update.
Os-install is the only exception. We should always install all bundles that are
listed as also-add in os-install.
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
This commit adds a few more cosmetic changes to the commands that run
verify in the back for consistency.
- Different steps in the update process are separated by a blank line.
- Messages from swupd should not finish with a '.'
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Currently when running diagnose, and a problem is found (a missing
file, a corrupt file, or an extraneous file), swupd shows the output
like this:
Verifying files
...12%
Hash mismatch for file: /usr/bin/somefile
...100%
Inspected 632545 files
So it splits the percentage that shows the progress of the action in two
lines. This commit fixes the output so it looks like this instead:
Checking for missing files
...100%
Checking for corrupt files
-> Hash mismatch for file: /usr/bin/mixer
-> Hash mismatch for file: /usr/bin/mixin
-> Hash mismatch for file: /usr/bin/swupd-extract
-> Hash mismatch for file: /usr/bin/swupd-inspector
-> Hash mismatch for file: /usr/share/zsh/site-functions/_mixer
...100%
Checking for extraneous files
...100%
Inspected 632545 files
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
This commit adds a line break at the end of the progress report (when we
reach 100%) so we don't have to manually include it every time we report
progress.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
The purpose of swupd verify --install is so it can be used to install
a new Clear OS in a target system. However since currently it is part
of the swupd verify command it makes it confusing as of to what its
purpose is.
This commit moves the swupd verify --install option to be a top level
command (swupd os-install) so its purpose is more straightforward. The
new command still uses the verify code to perform the task underneath.
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
Brett T. Warden