This adds a more convenient means to filter out most, but not all, development
dependencies.
* guix/build/node-build-system.scm (%dependency-keys)
(%dev-dependency-keys): New variables.
(delete-dependencies) [#:negate?, #:dependency-keys]: New arguments. Log
removed dependencies.
(delete-dependencies/except, delete-dev-dependencies/except): New procedures.
* tests/node.scm: New test.
* Makefile.am (SCM_TESTS): Register it.
Follow up of 8d27b1158d discussed in #9231.
* tests/git.scm ("update-cached-checkout, recursive submodules follow ref"):
Replace the nested 'let' by 'let*'.
Merges: #9683
Change-Id: Ibd212bbd2b6fbf02f6fcae0ecb21922686b5f8bc
This fixes a regression introduced in 26d7eb8a4a
when running ‘guix-daemon’ without root privileges.
On rootless installations, we must abide by the usual file access rules, which
include the requirement that a directory be writable before it be renamed (the
rationale being that its ".." entry needs to be modified).
* guix/scripts/substitute.scm (rename-file*): New procedure.
(guix-substitute): Use it.
* tests/substitute.scm (directory-nar-sha256): New procedure.
(%narinfo/directory): New variable.
(call-with-narinfo): Add '#:directory?' keyword argument.
(with-directory-narinfo): New syntax.
("substitute, authorized key, directory"): New test case.
Fixes: guix/guix#9686
Change-Id: I9d399dd5f1717b7d02854d97e1100a0a464fac75
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #9702
* guix/profiles.scm (build-system->manifest): New procedure.
* doc/guix.texi (Writing Manifests): Document it.
* tests/profiles.scm: Test it.
Merges: #7272
Change-Id: Iaba54d5d4a1a9ae304de45f91374098617adebfc
Signed-off-by: Simon Tournier <zimon.toutoune@gmail.com>
Previously, by serving a valid, signed, and authorized narinfo that does not
match what the client asked for, an attacker could cause users to get the
wrong substitute.
* guix/substitutes.scm (fetch-narinfos)[handle-narinfo-response]: Check
whether the result of ‘read-narinfo’ matches ‘request’ and ignore it if it
doesn’t.
[do-fetch]: Likewise with ‘narinfo-from-file’.
* tests/substitute.scm ("query narinfo that returns different store path")
("substitute, narinfo does not match requested store item"): New tests.
Reported-by: Reepca Russelstein <reepca@russelstein.xyz>
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #9665
Recent changes caused (guix narinfo) to start validating the StorePath of all
narinfos, which is what finally caught this.
* tests/challenge.scm (call-mismatch-test): produce an item path whose hash
has 32 nix-base32 characters, rather than 32 bytes.
Change-Id: I19a5c0106c913d9ad37fb53a28e8add8bd6ec2a0
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
This change safeguards against narinfos that would advertise
“StorePath: /gnu/store/…-foo/../../../etc/passwd” or similar, which could
otherwise propagate down the line and lead to attempts to overwrite said file,
should ‘guix-daemon’ be running as root.
* guix/narinfo.scm (narinfo-maker): Validate ‘path’, ‘references’, and
‘deriver’ and return #f upon failure.
(read-narinfo, string->narinfo): Adjust docstring accordingly.
* guix/substitutes.scm (fetch-narinfos): Remove check for ‘narinfo-path’
prefix; instead check whether ‘read-narinfo’ returned true.
(cached-narinfo): Check whether ‘string->narinfo’ returned true.
* tests/substitute.scm ("query narinfo that contains invalid store path"): New
test.
Change-Id: I0be2ea88b7f762144fca35373363364df3264b5a
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
The mistake was harmless because the ‘Deriver’ and ‘References’ field values
were unchecked so far.
* tests/substitute.scm (%narinfo)
("query narinfo with signature over irrelevant bits")
("query narinfo with signature over relevant subset")
("substitute, invalid hash")
("substitute, two invalid narinfos"): Remove store prefix from ‘Deriver’ field
value. Add hash part to ‘Deriver’ and ‘References’ values.
* tests/guix-daemon.sh: Likewise.
Change-Id: I0af4b9aa4b64cebf46d221ffab17bf5663b0cc11
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
The hash of a nar is only known once 'download-nar' completes. By that point
the nar may already have been partially-restored to its destination for some
time. Additionally, after a hash mismatch is detected, 'guix substitute'
leaves the invalid contents at the specified destination. This means that
untrusted, attacker-controlled contents are present at what at some point may
have been or at some point may become a valid store path. It is entirely
possible that a user or program won't check that a given store item is valid
before trying to use it (it may be that 'guix gc' was run while an obscured
reference was held). Let's protect them in that case by ensuring that
unverified contents are kept elsewhere, in a temporary directory.
While implementing this one such program was discovered: tests/substitute.scm.
It actually incorrectly computes the hashes of the nars it supplies to (guix
scripts substitute), computing the hash of the plain contents instead of the
hash of the nar. These tests nevertheless passed because they didn't check
what (guix scripts substitute) gave as output, instead only checking that the
expected contents were at the specified destination.
* guix/scripts/substitute.scm (call-with-temporary-directory-in): new
procedure.
(guix-substitute): use it to create a temporary directory next to the final
destination to restore the nar inside of. Once it is fully restored, if the
hash matches, move it to the final destination, deleting the file currently
there if it already exists.
* tests/substitute.scm (plain-file-nar-sha256): new procedure.
(%narinfo, "substitute, narinfo with several URLS" test): use it.
Change-Id: Ifeb9b4c912f9d9b2e7477d237da04e7d5175ff83
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
There are two potential places where file:// URIs could be used in 'guix
substitute':
1. The substitute urls, which are used for fetching narinfos. These can come
from the daemon, via daemon option "substitute-urls", from the client, via
daemon option "untrusted-substitute-urls", or from 'guix discover'. The
latter two must not be allowed to provide file:// URIs, since that could be
used to cause the daemon to read private files, files in /proc or /dev,
etc, and various parts of those files may show up in the backtrace from
'guix substitute'.
2. The narinfos themselves, which may offer arbitrarily many URIs to download
the nar of the store item in question. These URIs do not need to be
in the signed portion of the narinfo, so they may be attacker-controlled.
These should also not be allowed to provide file:// URIs, with the sole
exception being in the case of the test suite for the sake of simplifying
the tests.
To accomplish this, we error out of 'guix substitute' if
"untrusted-substitute-urls" includes a file:// URI, and we filter out all
file:// URIs from those provided by 'guix discover'. We also ignore all
narinfos containing file:// URIs unless '%allow-unsafe-substitute-uris?' is #t.
* guix/scripts/substitute.scm (%allow-unsafe-substitute-uris?,
%allow-unsafe-narinfo-uris?): new parameters.
(process-substitution, process-substitution/fallback): honor
'%allow-unsafe-substitute-uris?'.
(uri-safe?, narinfo-uris-safe?, assert-safe-uris): new procedures.
(%default-substitute-urls): use assert-safe-uris for the
"untrusted-substitute-urls" daemon option.
(%local-substitute-urls): filter out file:// URIs.
* tests/substitute.scm ("file:// URI prohibited by default"): new test. Set
'%allow-unsafe-substitute-uris?' to #t after it finishes.
Change-Id: I34d7e58b5a01c7cc61b52fcd61b547bba2ce4300
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Also validate that they are in strictly ascending order, which also ensures
that there are no duplicate names.
When 'guix substitute' fetches a nar, it does so with a validly-signed hash
already known from the narinfo. But it can't verify that the hash of the nar
it's currently fetching matches until the full nar is downloaded. Until then,
'download-nar' will extract the nar into the specified destination using
'restore-path', which avoids having to keep a file of unbounded size in
memory. Critically, this means that the input that 'restore-path' (and by
extension 'fold-archive') is processing is untrusted, since substitute server
TLS certificates aren't verified (the narinfo signatures are supposed to make
it unnecessary).
As such, the scope of harm that can be caused by a malicious nar being
processed by 'restore-file' needs to be minimized.
* guix/serialization.scm (valid-nar-file-name?): new procedure.
(fold-archive): Use it to verify that directory entry names do not contain
'/' or '\0', and they are not equal to ".", "..", or "". Also verify that
they are in strictly ascending order.
(call-with-port*): new procedure.
(dump-file): use O_EXCL and O_NOFOLLOW. This precaution ensures that even
if 'restore-file' is somehow tricked into writing to a symlink (not
currently believed to be possible), it will result in an error.
* guix/store/deduplication.scm (call-with-fresh-output-file): new procedure.
(dump-file/deduplicate): use it for similar reasons as 'dump-file'.
* tests/nar.scm (call-with-tree-port, port-bad-nar?): new procedures.
("write-file-tree + fold-archive, unsorted directory entries",
"write-file-tree + fold-archive, duplicate directory entries",
"write-file-tree + fold-archive, invalid directory entries"): new tests.
* tests/publish.scm (call-with-temporary-output-filename): new procedure.
("/nar/*", "/nar/gzip/*", "/nar/lzip/*", "/nar/zstd/*", "/nar/ with properly
encoded '+' sign"): use it in these test cases so that the output filename
doesn't name an already-existing file.
Change-Id: I41f248c13d7af787233afad5cae102056329a68b
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Previously, update-cached-checkout updated submodules before switching the
cached parent checkout to the requested ref. A second recursive update could
therefore update submodules according to the previously checked-out parent
tree, then reset the parent to a ref whose gitlinks name different submodule
commits, leaving submodule worktrees stale.
* guix/git.scm (update-cached-checkout): Update submodules after switching to the requested ref.
* tests/git.scm ("update-cached-checkout, recursive submodules follow ref"): New test.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Fixes: guix/guix#7741
Merges: #9231
Specifying all three of -K, --check, --rounds=3 caused valid packages to be
deleted from the store.
When `buildMode == bmCheck`, the built package is not renamed from
`actualPath` to `i.second.path. Then later, when `settings.keepFailed` and
`curRound < nrRounds`, `i.second.path` was renamed to `i.second.path + "-check"`
anyway.
* nix/libstore/build.cc (DerivationGoal::registerOutputs): Add check for bmCheck
before renaming store path.
* tests/store.scm ("build-things, check mode"): Ensure file is a valid path.
("build-things, check mode + keep-failed"): New test.
Fixes: #9631.
Change-Id: I31d24b2349f563867b8101a1b5ce3c10896ce165
Co-authored-by: Ludovic Courtès <ludo@gnu.org>
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #9632
Without running 'checkout' as the last Git directives to one branch that
doesn't own the Git tag, the test is trivial thus poorly relevant. This
change checks that 'update-cached-checkout' and that the 'cached-directory'
return both the correct Git object identifier matching the upstream Git object
identifier of the Git tag.
* tests/git.scm ("update-cached-checkout, tag"): Switch to 'master' branch and
improve the checks.
Fixes: guix/guix#9340
Merges: guix/guix!9341
Change-Id: I698f94858a4247b8bdcc54758f38f3623890b6af
The length of the repository url causes it to go on its own line in 'guix
build guix' but not in 'make check' from a regular checkout with a shorter
TMPDIR. Skip the tests in the case where they are currently known to fail,
until a proper fix can be implemented so that this doesn't block security
fixes from being deployed.
* tests/style.scm (%temporary-repository-file-name-too-long?): new variable.
("url-fetch->git-fetch, basic transformation",
"url-fetch->git-fetch, preserved field" test cases): conditionally skip.
Fixes: guix/guix#6946
Change-Id: I4c14a89dac042a556b829398d12242b67cf579cd
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #9623
(current-channels) is empty in a regular guix checkout, but 'guix build guix'
passes --with-channel-commit=..., which causes it to return the configured
guix channel. Consequently (trusted-channels) isn't empty and these tests
fail in 'guix build guix'.
* tests/guix-time-machine.sh: explicitly create an empty trusted-channels.scm.
Change-Id: I4584ee96f49edbad414cae611126679e9000e555
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Previously an authorized substitute server could produce invalid store paths -
that is, paths that do denote a top-level file in the store, but that do not
obey the syntax restrictions beyond what that implies. Given that an
authorized substitute server can already potentially do a lot of damage if it
really wanted to, this isn't a major issue, but closing off this opportunity
does simplify the analysis somewhat.
* nix/libstore/local-store.cc (LocalStore::importPath): use
strict readStorePath(s) variants.
* tests/store.scm ("import path not in store, unsigned", "import path not in
store, signed", "import invalid path, unsigned", "import invalid path,
signed" test cases): new test cases. The "not in store" cases succeeded
previously, while the "invalid path" cases did not succeed prior to this
commit.
Fixes: guix/guix#9078
Change-Id: Ib81c19ec1ae0fff5b7c7268f4f7429b16a870996
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #9434
This works around guix/guix#7690 whereby calling ‘primitive-fork’ from a child
process created with ‘safe-clone’ causes said child process to hang.
* tests/containers.scm ("call-with-container, pid namespace")
("container-excursion")
("container-excursion, same namespaces")
("container-excursion*")
("container-excursion*, /proc"): Skip.
Change-Id: Ieb49a5c5e3e92cff40f6dd2472d0cb177680d9f4
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Merges: #9085
Fixes a regression introduced in 94ae360ab4,
which left the test broken.
* tests/gexp.scm ("local-file, relative file name, within gexp, compiled"):
Remove reference to ‘module’ variable. Adjust second argument to ‘load*’.
Change-Id: If7889645a0d5b0678a92e4c2e009ae0f54df6bee
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* tests/pack.scm ("appimage + localstatedir"): Create file “OUTPUT” and then
copy it to #$output.
Change-Id: I73be47a336e46c7099b940a02b13f8192145a185
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Using ‘match’ to check requests is more conventional and more concise. It
also avoids relying on Guile-Lib, which is an optional dependency, for its
‘assert’ macro.
* tests/machine/hetzner/http.scm ("hetzner-api-actions-unit")
("hetzner-api-locations-unit")
("hetzner-api-server-types-unit")
("hetzner-api-server-delete-unit")
("hetzner-api-server-enable-rescue-system-unit")
("hetzner-api-server-power-on-unit")
("hetzner-api-server-power-off-unit")
("hetzner-api-server-reboot-unit")
("hetzner-api-ssh-key-create-unit")
("hetzner-api-ssh-key-delete-unit")
("hetzner-api-ssh-keys-unit")
("hetzner-api-primary-ips-unit"): Use ‘match’ instead of ‘cond’ and ‘assert’.
Change-Id: Idf8cb1c11b332ffe780ed83c624dc50484b0db52
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Fixes a regression introduced in f98c00ab74.
* tests/machine/hetzner/http.scm ("hetzner-api-server-create-unit"): Rewrite
using ‘match’ and add clause for /v1/images.
Change-Id: Ib87ebb48c6765b5228a24092a974bd76605a7288
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Verifies that when only one of multiple license strings is recognized,
the resulting list has the correct license followed by one instance
of the symbol 'unknown-license!
* tests/import/gem.scm (test-mixed-licenses-json): New fixture.
("gem->guix-package with mixed valid and gibberish licenses"):
New test.
Change-Id: I33a003d50ba20d30360b52c448ae28ad9d37233b
Merges: https://codeberg.org/guix/guix/pulls/8276
Reviewed-by: Carlo Zancanaro <carlo@zancanaro.id.au>
Signed-off-by: Nguyễn Gia Phong <cnx@loang.net>
Exercises the map in 4f78d33f0 where every license
string is unrecognized, so the resulting package should have
(license (list unknown-license! unknown-license!)).
* tests/import/gem.scm (test-gibberish-licenses-json): New fixture.
("gem->guix-package with gibberish license identifiers"): New test.
Change-Id: I85cbb0574f9393749a84f4cf5818d4c1083a8476
Reviewed-by: Carlo Zancanaro <carlo@zancanaro.id.au>
Signed-off-by: Nguyễn Gia Phong <cnx@loang.net>
Similar to what was done for the crate importer in 263a267b75.
* guix/import/gem.scm (string->license): Try spdx-string->license first,
returning symbols instead of license objects. Fall back
to the existing hardcoded table for non-SPDX strings.
(make-gem-sexp): Remove license->symbol calls;
licenses are now symbols directly.
* tests/import/gem.scm (test-spdx-json): New fixture.
("gem->guix-package with SPDX license identifiers"): New test.
Change-Id: I5fa0ebdfd3a3393eab5ef0554ed09887a89bc107
Reviewed-by: Carlo Zancanaro <carlo@zancanaro.id.au>
Signed-off-by: Nguyễn Gia Phong <cnx@loang.net>
The RubyGems API returns "homepage_uri": null for gems with no homepage.
guile-json parses JSON null as the Guile symbol 'null', which passes
through into the generated package sexp as (home-page null)—an unbound
variable error at evaluation time.
* tests/import/gem.scm (test-no-homepage-json): New fixture.
("gem->guix-package, bald homepage_uri"): New test.
* guix/import/gem.scm (non-empty-string-or-false): New procedure.
(<gem>)[home-page]: Use it.
* guix/import/utils.scm (non-empty-string-or-false):
New exported procedure, moved to here...
* guix/import/gem.scm (non-empty-string-or-false): ...from here...
* guix/import/pypi.scm (non-empty-string-or-false): ...and here.
Change-Id: If8f2ca32834d762c753797067ef87937503ff9f9
Reviewed-by: Carlo Zancanaro <carlo@zancanaro.id.au>
Signed-off-by: Nguyễn Gia Phong <cnx@loang.net>
After this commit, when both modules are imported:
- if pyproject is imported before, the warning is 'pypi-uri' is
deprecated, use '(@ (guix build-system pyproject) pypi-uri)' instead
- if python is imported before, the warning is `pypi-uri' imported from
both (guix build-system python) and (guix build-system pyproject)
This seems convenient enough to warn for deprecation in the short term,
while avoiding any breaking changes.
* guix/build-system/pyproject.scm (pypi-uri): Move the procedure from
(guix build-system python) here.
* guix/build-system/python (pypi-uri): Drop definition, import it
from (guix build-system pyproject) and deprecate it.
* gnu/packages/openldap.scm: Reorder modules to get the right warning.
* gnu/packages/pypy.scm: Likewise.
* gnu/packages/*.scm : Drop module (guix build-system python).
* tests/import/pypi.scm: Likewise.
Merges: https://codeberg.org/guix/guix/pulls/7448
Change-Id: Ib42f53bc545052eb7918a25afe9db6d5fc2cb834
Reviewed-by: Nguyễn Gia Phong <cnx@loang.net>
Signed-off-by: Sharlatan Hellseher <sharlatanus@gmail.com>
* guix/build/pyproject-build-system.scm (set-version): Ensure source
is not #f before using it.
* tests/builders.scm
(make-python-dummy)[build-system]: Switch to pyproject-build-system.
[native-inputs]: Add python-setuptools.
(python-dummy-no-setuptools): Drop it.
(check-build-success, check-build-failure): Refresh tests.
(check-build-failure): Drop python-dummy-no-setuptools.
Change-Id: I892b45c34b506ff27634e2ef706009dc81e831ec
Signed-off-by: Sharlatan Hellseher <sharlatanus@gmail.com>
This fixes a regression introduced in commit a2ef2bcbfd ("machine: hetzner:
Allow connections using ssh-agent"), where having no ssh-key would cause the
early Hetzner provisioning to fail due to the lack of SSH authentication.
* gnu/machine/hetzner.scm (%hetzner-ssh-key-file): New parameter.
(<hetzner-configuration>) [ssh-public-key]: Compute default according to the
value of the private ssh-key field. Introduce an indirection via...
(hetzner-configuration-ssh-public-key): ... this new procedure, to honor
%hetzner-ssh-key-file.
(hetzner-configuration-ssh-key): Likewise for the private ssh-key.
(hetzner-configuration-ssh-key-fingerprint): Rename to...
(public-key->fingerprint): ... this, taking just the public key as argument.
Update doc.
(hetzner-configuration-ssh-key-public): Rename to...
(public-key->string-with-type): ...this, for clarity, and accept just a
public-key object. Update doc.
(hetzner-machine-ssh-key): Exit early with #f when no ssh key is provided.
(temporary-ssh-key-file): New procedure.
(hetzner-machine-ssh-key-create): Assert a public key is defined. Adjust
calls to renamed procedures, and adjust for the `hetzner-api-server-create'
API change.
(hetzner-machine-enable-rescue-system): Refine doc and fix code path when
ssh-key is #f.
(hetzner-machine-rescue-install-os): Document.
(hetzner-machine-provision): Refine doc. Remove extraneous sleep and #:unwind
argument. Limit ssh-session dynamic scope to where it's needed.
(cleanup-temporary-ssh-key/maybe): New procedure.
(deploy-hetzner): Create and use a temporary SSH key when none is defined, and
clean it up when done or on errors.
* gnu/machine/hetzner/http.scm (hetzner-api-server-create): Make ssh-keys a
keyword argument, and fix execution when it's #f.
(hetzner-api-server-enable-rescue-system): Likewise.
* tests/machine/hetzner.scm ("deploy-machine-mock-with-unprovisioned-server"):
Adjust test.
* tests/machine/hetzner/http.scm (create-server)
("hetzner-api-server-create-unit")
("hetzner-api-server-enable-rescue-system-unit")
("hetzner-api-actions-integration")
("hetzner-api-server-enable-rescue-system-integration"): Likewise.
* doc/guix.texi (Invoking guix deploy): Precise what happens when 'ssh-key' is
#f in hetzner-configuration, and suggest declaratively authorizing your SSH
key.
Change-Id: I812b348fb553f3b5aebd0bf66850c6ecb9e06653
dhcp-client-service-type has been deprecated for a while as the
ISC implementation has reached end-of-life upstream in 2022. This
is the last remaining user of dhcp-client-service-type in the Guix
repository.
* tests/guix-system.sh: Use dhcpcd-service-type.
Change-Id: Iaf8dd4bc888c99815d2d6c8c195fe92644186625
* guix/swh.scm (swhid-content-data): New procedure.
(call): Do not close ‘port’ when ‘result’ is the same as ‘port’.
* guix/scripts/pull.scm (swhid-content-data*): New procedure.
(channel-list): Accept ‘file’ as a SWHID.
* tests/guix-time-machine.sh: Add test.
* doc/guix.texi (Invoking guix pull): Document it.
(Invoking guix time-machine): Likewise.
Change-Id: I8145cd8685fe2926b1548d4a2dcd54804d89228a
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
* guix/ui.scm (load/isolated): Change ‘file’ parameter to ‘port’ and adjust
accordingly.
(load*): Change ‘file’ to ‘file-or-port’ and adjust accordingly.
* tests/ui.scm ("load/isolated, reading exceeds limits")
("load/isolated, attempt to import module")
("load/isolated, attempt to allocate with 'cons'")
("load/isolated, attempt to allocate with 'make-vector'")
("load/isolated, use of allowed bindings"): New tests.
Change-Id: I0ec8fa2717c02041d409f6dc59b753d4501107f9
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
Dariqq