974 Commits
Author SHA1 Message Date
Otavio Pontes a9c1366019 Release v3.18.5
This release adds enhancements, bug fixes and test improvements.

Enhancements:
 - Remove dependency on verifytime binary
 - Prettier --time outputs

Bug Fixes:
 - Don't fail an update if a bundle is removed on server
 - Inform out of disk errors on download failures
 - Fix TOCTOU problem when verifying system time
 - Adjust system time on check-updates to assure signature
   validation will work
 - Fix error on output of verifytime when errors occurred.
 - Behavior of verify and verify --fix is know consistent
 - Fix incorrect --time output on bundle-add
 - Use correct permissions when creating parent for statedir directory
 - Apply system locale on swupd to avoid warnings on filename conversions
 - Don't affect autocomplete behavior on a verify --fix

Tests
 - Adding new tests and improvements to existing tests
 - Full tests reorganization to have a better view on the total
   functional test coverage

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
v3.18.5
2018-11-28 20:58:18 +00:00
Otavio Pontes dd9355227b Don't save the latests Manifest as a hidden file
As we aren't saving the MoM in a directory we were reading to look for
bundles, we don't need to save it as a hidden file.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-26 17:42:57 -08:00
Castulo Martinez ff175e438e Removing update-apply-full-file-delta as duplicate
One test was removed: update-apply-full-file-delta.bats (it was a
duplicate of test/functional/update/update-use-pack.bats)

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-26 16:04:42 -08:00
Castulo Martinez 80e9683a04 Adding IDs to update tests
Adding the IDs to the update tests and some minor improvements.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-26 16:04:42 -08:00
Otavio Pontes 914048fb4a hash: Use snprint instead of sprintf to prevent overflows
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-26 14:24:25 -08:00
Otavio Pontes 7d6ea4bbe5 search: Prevent overflows by not using strcpy
Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-26 14:24:25 -08:00
Castulo Martinez 1fbfe1c2ef Remove old test files from gitignore
Removing some files that are no longer applicable from the gitignore
file.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-26 22:16:30 +00:00
Castulo Martinez 629b2fe9e2 Move the MoM from bundles dir to /var/tmp/swupd
swupd copies the latest MoM to /usr/share/clear/bundles/.MoM so we
can use it for bash completion. The problem is that this directory
is tracked by clearlinux and because of that "swupd verify --picky"
will list this file as an extra file. And "swupd verify --fix --picky"
will remove that file.

This commit fixes the issue by changing the location where the .MoM
is copied to being this an untracked location, so the file won't be
removed.

Closes #632

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-26 14:16:00 -08:00
Otavio Pontes 18c49d3d02 locale: Set locale to system locale
Standard locale on C applications is C and we use in clearlinux UTF-8
for file names. Libarchive checks the program locale to convert
filenames so we need to set this.

According to setlocale documentation, setlocale(LC_ALL, "") sets the locale
to system default (in the case of ClearLinux, UTF-8).

More information on the libarchive problem:
https://github.com/libarchive/libarchive/wiki/Filenames and
https://github.com/libarchive/libarchive/issues/587

Fixes #445

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-26 14:13:45 -08:00
Castulo Martinez 46ea5954fb Removing option --force where is not needed
The --force option is included in bundle-add, bundle-remove,
check-update, update and verify, but it is really only used in
verify.

This commit removes the option from the commands that are not using
it so it is not misleading to users. If at some point we need to
implement somthing with --force it can be added back at that point.

Closes #636

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-20 12:31:14 -08:00
Castulo Martinez 38dddc09a1 Adding ID to search tests
Adding ID to search tests and changing the description of some of
them to make them clearer.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-20 10:03:49 -08:00
Castulo Martinez 41d334e708 Adding IDs to hashdump tests
Adding IDs and minor changes in hashdump tests.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-19 12:47:23 -08:00
Castulo Martinez e847ad89c9 Adding IDs to usability tests
Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-19 12:46:46 -08:00
Castulo Martinez 09967f5508 Adding IDs to mirror tests
Adding IDs to mirror tests and some small changes.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-16 15:51:32 -08:00
Castulo Martinez a615bd17f8 Updates bundle-add -t test to include tree view
Updates test ADD030 to include the time in tree like view.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-16 15:50:26 -08:00
Castulo Martinez 8b715c9ac1 Prettifying the --time output
Changing the way time is displayed when using the --time option so it
shows as a tree view so it is easier to see what items are sub-tasks
of other tasks.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-16 15:50:26 -08:00
Castulo Martinez 986fcd5604 Continue check-update when time is not correct
When the system time is way off, the certificate validation will
fail. For this reason, many swupd commands attempt to fix the
system time if wrong. check-update was not doing this so the
command would fail in this situation.

This commit adds a verification for this so check-update attempts
to fix the system time and continue instead of failing.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-16 14:31:11 -08:00
Otavio Pontes caa5e2a43c verifytime: Fix return value of verify_time function
There was some confusion about shell return code and bools.
Shell returns 0 for true and bools are 0 for false.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-16 13:16:34 -08:00
Otavio Pontes 6ff94a82df helpers: Don't enforce umask on mkdir_p, just force that when needed
Swupd state dir should be masked as 700, but we can't mask all parent directories
as 700. This is particularly a problem because /var and /var/lib/ shoudn't
be 700, but /var/lib/swupd should. So creating all folders with current
umask and using chmod syscall to set only the swupd statedir diretory to
root only

Also, stop using mkdir_p where are no longer needed

Fixes #655

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-16 10:44:28 -08:00
Reagan Lopez 5cb094e1b1 test: Test install time option for bundleadd
Signed-off-by: Reagan Lopez <reagan.lopez@intel.com>
2018-11-16 10:09:39 -08:00
Castulo Martinez 14df661f27 Adding ID to check-update tests
Adding ID to check-update tests and changing their description to
make it more readable.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-16 10:05:45 -08:00
Castulo Martinez 5e438b3831 Adding ID to bundle-list tests
Adding an ID to all bundle-list tests and rewording some of the
test descriptions to make it more readable.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-16 10:05:14 -08:00
Otavio Pontes 599fec9419 xattrs: Create dummy implementations for xattr API when disabled
When xattr is disabled in a build, use a dummy implementation of the API
instead of counting on dummy implementation of the functions used by xattr.
By doing that we are clearer on the output of the functions, less code
executions and we get rid of a dead code.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-16 08:25:43 -08:00
Otavio Pontes 4b16673136 Link swupd with verifytime
Instead on counting with the verify time in the system, link swupd
with verifytime lib.

This reduces one point of failure of swupd on recovering a corrupt system.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-16 08:22:14 -08:00
Castulo Martinez 8bd01ff025 Clean up unused variables from test
There are some variables left over from commit
897f5d44c8

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-16 08:21:19 -08:00
Castulo Martinez fdaa46d0f4 Adding ID to install tests
Some verify --install tests were submitted some time ago before we
started using IDs for tests, so these tests are missing the ID.

This commit add those missing IDs and make a couple of improvements
in the tests.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-15 10:54:29 -08:00
Castulo Martinez 4c2b636f07 Correct test group code for checkupdate
The check-update tests should use a group code of CHK, but the
test library was using AUT incorrectly which belongs to the autoupdate
tests.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-15 10:51:19 -08:00
Castulo Martinez 8f48252e41 Fix bug when showing times in bundle-add
Option --time/-t can be used with bundle-add to show verbose time
output for swupd operations. At the moment it is showing the info
incorrectly. It is showing the data twice and the times showns are
off.

This commit fixes the bug.

Closes #675

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-14 14:00:48 -08:00
Geoffroy Van Cutsem 047c61b1fc verifytime: modify set_time() to take single argument
Modify the set_time() function used in verifytime.c to take a single
argument instead of two. The second argument that was previously used
can easily be derived from the first and it avoids situations where
these two are not coherent (which happens to the case in the current
version of verifytime.c)

Signed-off-by: Geoffroy Van Cutsem <geoffroy.vancutsem@intel.com>
2018-11-14 13:10:01 -08:00
Castulo Martinez 26a748394f Adding IDs to bundle-remove tests
This commit adds the test ID to every bundle-remove test, removes
duplicates and make readability improvements.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-14 13:09:42 -08:00
Castulo Martinez 21af288b4f Adding tests for verify --picky
This commit adds some missing tests for "verify --picky", the rest
of the tests were added recently as part of other commits.

This commit also add the test ID to every verify test, and remove
duplicates.

Closes #315

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-13 23:17:14 +00:00
Castulo Martinez d8c1984c7c Add consistency between verify and verify --fix
Running "swupd verify" should be a dry-run of running
"swupd verify --fix", meaning that "swupd verify" should find
the same issues to be resolved than "swupd verify --fix" but it
should just inform about them while --fix should inform + fix them.
In occasions this does not happen, for example when there are
tracked files that need to be removed from the system.

This commit fixes that.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-13 13:56:01 -08:00
Castulo Martinez 1ad0fc7778 Tests for verify and verify --fix consistency
This commit adds some tests to validate consistency
between "swupd verify" and "swupd verify --fix".

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-13 13:56:01 -08:00
Otavio Pontes b72c01bbbe verifytime: Fix TOCTOU problem
We can't assume that the file wasn't deleted between the execution of a
stat and a fopen. Printing error only when fopen fails.

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-13 13:22:09 -08:00
Otavio Pontes e121a6fc19 archive: Ignore return code of function
Ignore function return code as we don't need to check if operation is
fatal or not at this point.
2018-11-13 13:22:00 -08:00
Brian J Lovin fa09d679cc Output hints to check free space on download failures
Currently some of the output from swupd can be cryptic when
files fail to download. Add a hint to the user to check
their free space if Curl returns write errors.

Closes #486

Signed-off-by: Brian J Lovin <brian.j.lovin@intel.com>
2018-11-13 21:19:24 +00:00
Castulo Martinez 660a9a34c4 Continue update when a bundle was removed upstream
If a bundle happens to be removed from the content server (or mix) it
means the bundle won't be in the MoM anymore, so the bundle in the
system will look like an invalid bundle. When this happens the update
is currently stopping.

This commit fixes this so If this happens, the user is informed, and
the update continues.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-12 14:54:23 -08:00
Castulo Martinez 73b3269a1b Test for update when a bundle was removed upstream
Adding an update test for the case when a bundle is removed upstream
from the content server (or from a mix).

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-12 14:54:23 -08:00
John Akre 6186a177c8 test: clean up tests when interrupted
Tests will be cleaned up by a trap routine when interrupted early. This
change also moves test web server and certificate store clean up to the
destroy_test_environment function.

Signed-off-by: John Akre <john.w.akre@intel.com>
2018-11-12 14:30:45 -08:00
John Akre 1640fadc8a test: Move test files out of /tmp
Some tests wrote files and directories to /tmp. This change moves the
test certificates directory into the swupd repo's top level directory
and moves other test files into their corresponding test directories.

Fixes #650

Signed-off-by: John Akre <john.w.akre@intel.com>
2018-11-12 14:30:45 -08:00
Otavio Pontes 905c0495d3 test: Moving verify --install tests to verify folder
Also:
 - remove unnecessary mkdir lines from tests
 - Check if /core file is present in install-multiple test

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-12 22:20:58 +00:00
Matthew Johnson 316b42d327 tests: add verify --install tests
This adds several tests for os installation functionality.
* install - regular installation of just os-core
* install-multiple - install os-core and a test-bundle
* install-bad - attempt to install an invalid bundle name
* install-no-zero-packs - install os-core and test-bundle with no
  zero-packs available. Fullfiles still available as a fallback.
* install-no-fullfile-fallback - attempt to install with no zero packs
  and no fullfiles available for fallback.

Signed-off-by: Matthew Johnson <matthew.johnson@intel.com>
2018-11-12 22:09:46 +00:00
Otavio Pontes 76c5ca0bb9 Release v3.18.4
This release fixes some bugs and add tests improvements.

Bug fixes:
 - Fix support for openssl 1.1
 - Bash completion for bundle-remove wasn't complete
 - Print error message when disk is full on a tarball extration

Tests
 - Running static analysis of all bat tests automatically and reported problems
   were fixed.
 - Test Makefile was problematic and some tests were skiped
 - More tests added

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
v3.18.4
2018-11-12 19:39:00 +00:00
Otavio Pontes 63b8ce2623 curl: Don't warn about curl fd leaks
Curl isn't closing urandom and random when using openssl 1.1. As it's something
out of our control and this won't create any resource leak problems, we're not
reporting that anymore.

This fd leak report is triggered when any https download is performed by swupd

Signed-off-by: Otavio Pontes <otavio.pontes@intel.com>
2018-11-09 15:03:22 -08:00
Tudor Marcu 0f26c78c42 Fix purpose in store verification
The purpose must be set explicitly now for the verification to
succeed, and in our case we must use "any" as defined in the generated
certificate purpose.

Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
2018-11-09 12:52:31 -08:00
Tudor Marcu 131252f1f3 Update signature verification for OpenSSL 1.1
The ERR_remove_thread_state() function is deprecated as the thread
handling and many memory management operatoins have been re-written to
be internally handled by the API. The error code should be retrieved
by the proper function now from X509 store contexts.

Signed-off-by: Tudor Marcu <tudor.marcu@intel.com>
2018-11-09 12:52:31 -08:00
Brian J Lovin ffead6e434 archives: Add output for out of space errors
Libarchive's archive_error_string() method returns a textual
error message suitable for display.
However, this method returns only "Write Failed" if libarchive
runs out of hard disk space.

Check specifically for ENOSPC and print a correct
warning message to the user.

Signed-off-by: Brian J Lovin <brian.j.lovin@intel.com>
2018-11-07 13:41:08 -08:00
Castulo Martinez fce640d14d Adding a missing line break in the verify command
There is a missing line break that shows up When running
the "swupd verify" command and there is a hash mismatch.
The output looks like this:

$ sudo swupd verify
Verifying version 25590
Verifying files
	...1%Hash mismatch for file: /usr/bin/tmux
	...100%
Inspected 292278 files
  1 files did not match
Verify successful

This commit fixes the issue by adding a line break at the
beginning of the output.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-05 10:15:59 -08:00
Castulo Martinez 765664a5e6 Fixing a bug in the test list in Makefile
The list is missing a "\" at the end of one test which is causing
a group of tests to be ignored by the makefile, therefore they are
not being run.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-05 09:11:59 -08:00
Castulo Martinez 1438148bbd Adding functions to get test IDs in testlib
Every test should include a unique ID, so it is necessary to have
a way of knowing what is the next available ID for the group of
tests.

This commit helps with that issue by implementing the following
features in the test library:
 - Adds the function get_test_list which can be used to print a
list of tests in a specific group directory.
 - Adds the function get_next_available_id which return the next
test ID that can be used in a new test.
 - When using the generate_test function to generate an empty test
the next available ID will be used for the template.

This commit also adds the ID to a bundle-add test that was missing it.

Signed-off-by: Castulo Martinez <castulo.martinez@intel.com>
2018-11-02 15:25:09 -07:00