1833 Commits
Author SHA1 Message Date
Otavio Pontes 41f7cc6c9c Release v3.24.3
Bug Fixes:
 - On 3rd-party add, if the execution was aborted by user, we could end up with an invalid repository entry
 - Don't create hardlink for symlinks because this is not supported by some toos, like bsdtar

Enhancements:
 - Improving debug output on external commands executed, like scripts
 - Using same curl reporting callback for spinner and progress

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
v3.24.3
2020-03-20 19:13:27 -07:00
Castulo Martinez 01173b5279 3rd-party:bug_fix: don't add unreachable repos
When adding a 3rd-party repo, if the repo cannot be added it is reverted
back. However if a user uses an incorrect URL that hangs and the user
cancels the process, the repo add is never rolled back, which will cause
the invalid repo to be left in the system.

This commit fixes the issue by making sure the repo is reachable and
somewhat valid before adding it to the config file.

Closes #1384

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-20 14:31:24 -07:00
Otavio Pontes 233d4c7e9d Staging: Don't create hardlinks when installing symlinks
Alternative solution to solution proposed on #1318

Instead of trying to create a link just don't try to link symlinks.
Tested patch using bsdtar and implemented automated testing to
validate solution.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-19 12:52:34 -07:00
Otavio Pontes 5ebd1e3033 shellcheck: Add job to run shellcheck all on all merges on master 2020-03-19 09:53:39 -07:00
Otavio Pontes 82792caa71 shellcheck: Fix newly reported bash problems 2020-03-19 09:53:39 -07:00
Otavio Pontes 4eab2e54ad shellcheck: Don't stop shellcheck-all script on first error
This is script is very slow, so it's better to gather all information
before aborting.

Also improve the output of the test when an error happens
2020-03-19 09:53:39 -07:00
Otavio Pontes b3e18c8325 curl: Using same progress function used on async to sync downloads
Moving from PROGRESS function to XFERINFO that is recommended by the
manual.

This fix a warning in the build, so we can turn on -Werror in tests

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-19 08:55:02 -07:00
Otavio Pontes 0dbb561c6b curl: Use correct parameter to set XFER data
It looks like CURLOPT_PROGRESSDATA is working, but the correct parameter
to use for XFERINFO is CURLOPT_XFERINFODATA. There's no guarantee this will
work in the future, so porting to the recommended parameter.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-19 08:55:02 -07:00
Otavio Pontes 07d2a48039 tests: On mirror tests don't use possible real server names
On proxy envs we may need to wait for a timeout before proceeding and
this can slow tests down.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-18 15:51:27 -07:00
Otavio Pontes a16ef453c6 sys: Improve debug when running external scripts
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-17 14:01:10 -07:00
Otavio Pontes 80a0a3676a scripts: Support parameters on build_and_run
Add 2 parameters, one for numbers of jobs and other to skip tests execution

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-17 13:32:14 -07:00
Otavio Pontes 422cd13df8 script: Check for third-party flag after build
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-17 13:32:14 -07:00
Otavio Pontes db08520ed9 scripts: Don't use FORTIFY_SOURCE when optimizations are disabled
Both arguments were incompatible and warnings were being displayed for
this combination

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-17 13:32:14 -07:00
Otavio Pontes 6afe772d7e scripts: Don't add system CFLAGS on build script
The reason we have a build script is that we wan't to control all
configuration arguments, so don't add any implicity CFLAGS

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-17 13:32:14 -07:00
Otavio Pontes 2e332d479d globals: Set function as static
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-16 13:49:45 -07:00
Otavio Pontes 616878929e scripts: Add a script to run swupd basic tests
Besides having a script that is used to run tests on github
actions CI, adding another script that can be used in any
system to build swupd with different configurations and
run all needed tests.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-13 14:49:57 -07:00
Otavio Pontes 5986556a1f Release v3.24.2
Bug Fixes:
 - 3rd-party update could report some incorrect file permission changes due to symlink derreference
 - os-install --bundles with only os-core parameter was failing
 - Fix some invalid examples on swupd manual
 - Fix warning message on autocomplete for zsh

Tests and clean-up:
 - Funcional tests running in paralell for default and in lexicographic order
 - Removing code on parser and tests related to the deprecated iterative manifest
 - 3rd-party enabled by default on build systems
 - Printing 3rd-party compile flag status on swupd -v

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
v3.24.2
2020-03-13 11:11:41 -07:00
Castulo Martinez 44e595691c Bug fix: os-install --bundles with os-core fails
When doing a system install with "os-install --bundles" if the list of
bundles to install includes only "os-core" the install fails.

This commit fixes the issue.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-13 10:03:32 -07:00
Lucius Hu 6ab598c89e Fixed bug
Huge amount of changes are due to re-formatting. There are only four actual
changes:

- Added a ')' on line 203, 211, and 219.
- Changed line 280 from
  `'(help --status)'[Show the installation status of the listed bundles'`
  to
  `'(help --status)--status[Show the installation status of the listed bundles]'`
2020-03-13 09:54:16 -07:00
Otavio Pontes 9c85b275a3 docs: Replace vi with vim and os-core for xterm
Vi isn't a bundle anymore. Replacing all examples to vim.
os-core isn't a good bundle for example. Using xterm.

Related to #1369

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-13 08:04:37 -07:00
Otavio Pontes f130b303a9 build_opts: Print third party build info on swupd -v
We print all flags selected on build time on swupd -v. This is useful
to check if an specific swupd binary has one specific feature enabled.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-12 15:37:28 -07:00
Castulo Martinez 4d10f19459 Updating testlib README
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-10 12:12:00 -07:00
Castulo Martinez e9b229bcde Remove iterative manifest support from testlib
The iterative manifest effort was droped so this commit removes the
iterative manifest support from testlib.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-10 11:09:46 -07:00
Otavio Pontes 57e42323bf signature: Dummy signature_verify_data() function missing
When building swupd with --disable-signature we need to have all
signature functions with a dummy implementation that doesn't validate
the signature.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-10 11:00:43 -07:00
Otavio Pontes 96234f998d 3rd-party: Make 3rd-party default in the build
Instead of requiring an --enable-third-party, as 3rd-party was already
released, we can make it default.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-10 10:57:31 -07:00
Otavio Pontes 2f6370f40d manifest_parser: Don't ignore iterative manifests
We had a format bump and now Iterative manifests aren't supported
anymore. We are safe to remove that from the parser.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-10 10:43:00 -07:00
Otavio Pontes 4ab5677570 clean: Fix comment about iterative manifests 2020-03-10 10:43:00 -07:00
Castulo Martinez d3cf8ac1e1 Do not follow links when validating staged files
When updating 3rd-party bundles, swupd validates the files to be updated
to make sure there were no changes in file permissions that could be
risky for the system, it uses stat to get the file permissions. The
problem is that stat follows links, and while the files are in staging,
the links are really pointing to invalid locations so stat will always
fail.

This commit fixes the problem by not following links when validating
file permissions.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-10 10:19:22 -07:00
Otavio Pontes fa86651c9f Makefile: Be consistent in order of make check-functional
Sorting the bats test list to be consistent when running make-functional.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-10 10:17:34 -07:00
Otavio Pontes 34469dbbd5 Makefile: Propagate the --jobs flag to check-functional
Check-functional wasn't being run in parallel. Using $(MAKE) instead
of make to propagate make parameters

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-10 10:17:34 -07:00
Otavio Pontes 84e507d3f5 service: Remove check-update service
Clear Linux isn't using the check-update service anymore for a long time
and there's no other known uses for those scripts. So there's no reason to
keep them.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-09 14:10:51 -07:00
Otavio Pontes 2f86cbc7e3 docs: Reset release notes draft
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-09 10:56:05 -07:00
Otavio Pontes f1b720e16a Release v3.24.1
This release of swupd includes a swupd 3rd-party info command and
fixes some minor bugs on swupd 3rd-party error handling.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
v3.24.1
2020-03-08 14:11:54 -07:00
Otavio Pontes 126fff4e0d 3rd-party: Fixing some error handling problems on 3rd_party repo
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-06 16:58:14 -08:00
Otavio Pontes b6dc104462 Release v3.24.0
This release of swupd introduces management of 3rd-party repositories and
content installed from them. A 3rd-party repository enables the distribution
of user produced content. Multiple repositories can be managed along with the
default upstream server. Most of the swupd subcommands used for managing
upstream content are supported to manage 3rd-party content along with most of
their options.

Besides that, this release also adds multiple improvements in pre-existing
swupd commands and bug fixes.

Enhancements:
 - Adding flag --recursive to bundle-remove so dependencies can also be removed from the system as long as nothing else still depends on them
 - Improve the output of bundle-info and bundle-list to inform if dependencies were included or added by also-add flag
 - Reporting the total space cleared by swupd clean command
 - Display which bundles are installed on the output of bundle-info command
 - Run ldconfig on chroot when running swupd using --path parameter
 - Display a spinner before downloading a file to improve progress report
 - Support searching files on specific version on search-file command
 - Make swupd config files case insensitive

Bugs:
 - Invalid output when using an empty file as swupd config file
 - Memory leaks on initialization erros or downloading a file to a full disk
 - Bundle info was showing an incorrect size for large bundles
 - Number of files added by bundle-add was corrected

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-06 16:57:38 -08:00
Castulo Martinez d949305082 Reordering 3rd-party subcommands in help
Reordering so it matches the order of the regular swupd commands. Also
adding missing values to config file.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-06 15:08:34 -08:00
Castulo Martinez 68de707c04 Implementing the 3rd-party info command
Implementation of the 3rd-party info command to show the version of the
3rd-party repository along with the update URL.

Closes #1354

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-06 14:54:23 -08:00
Otavio Pontes aeb026db1c Revert "staging: don't convert symlinks to hardlinks"
This reverts commit be1deab965.

Castulo found a critical bug introduced by this commit. Reverting while we are
investigating the root cause and proposing a solution
2020-03-05 13:25:21 -08:00
Otavio Pontes be129d9c7d time: stop using deprecated function stime()
stime is deprecated in newer versions of glibc, so using the recommended
clock_settime() instead.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-05 12:07:47 -08:00
Castulo Martinez 1234a4a400 Adding draft for the 3rd-party release notes
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-04 16:03:32 -08:00
Castulo Martinez 22f31365e7 Forcing the removal of corrupt 3rd-party repos
This commit allow users to force the deletion of corrupt 3rd-party repos
which will end up deleting most of the repo's files, except for the
scripts that exported files from that repo.

Closes #1343

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-04 16:03:23 -08:00
Otavio Pontes 087b1d2514 sys: Prefer to use api from sys than using stat directly 2020-03-04 12:04:05 -08:00
Otavio Pontes 591cc3308b sys: Add function to check if a path is a dir without following symlinks
Also rename the is_dir() to a name similar to other functions that also follow
symlinks

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-04 12:04:05 -08:00
Castulo Martinez 215daa49b3 Do not apply heuristics to 3rd-party files
When adding an upstream bundle, we apply some heuristics to the bundle
files and based on that some files are skipped from being installed.
These heuristics don't apply to 3rd-party bundles.

This commit skips running heuristics when adding 3rd-party bundles.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-03-03 16:40:14 -08:00
Otavio Pontes 64872d13d9 tests: Remove test leftovers when the directory is incomplete
Sometimes a test fails before finisihing to create all the required
directories under the testdir. So destroy_environment thinks this is
not a valid test environment.
Touching a dot file to show that the directory is a test environment
instead on counting on its content.

Fixes bug #1135

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-03-02 15:22:00 -08:00
Castulo Martinez 4277580540 Removing extra step from 3rd-party remove
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-02-28 14:26:40 -08:00
Castulo Martinez 18a793fd60 Use of --assume and --yes flags to confirm a cert
When users add a 3rd-party repository they are asked if they want to
trust the certificate, this commit makes possible to use the
--assume flag to answer that question.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-02-28 14:16:11 -08:00
Otavio Pontes db8e559e38 scripts: Always run ld_config before running any update scripts
It's possible to create a post_update script to run ld_config, but we
had problems with this in the past, so it was decided to hardcode that
on swupd instead of relying on scripts.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2020-02-28 12:56:22 -08:00
Castulo Martinez 81fcc2872b Rewording the 3rd-party menu details
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-02-28 11:53:32 -08:00
Castulo Martinez 5b1250dcbf Multi-repo operations skip bad repo
When running an operation that should run in all the 3rd-party repos
installed, if one repo fails to set up, the process is aborted and the
operation is never performed in the repos after the failed repo.

This PR changes the proces so if one repo fails to set up it is just
skipped so the operation is attempted in the other repos.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2020-02-27 16:51:28 -08:00