- Use the same mechanism (debug_map) in Pal/Linux and Pal/Linux-SGX.
Previously, Pal/Linux emulated the _r_debug structure, normally
maintained by ld.so, but that cannot be done in SGX outer PAL,
because it's loaded by ld.so already.
- Maintain the debug maps outside of SGX enclave. This allows
initializing them before enclave start, and potentially makes
them easier to use.
- Initialize PAL debug map before enclave start. Previously, this
was done from inside the enclave, so you couldn't set a
breakpoint too early (e.g. in pal_linux_main).
- Store only load address, without list of sections. This is to
avoid parsing the list of sections just to report them to the
debugger. Unfortunately, the GDB version that we support still
needs these sections, but we can retrieve them in GDB plugin.
- Move Python GDB code related to debug maps to a common file.
This is the next part of the great loader rework, with a lot of breaking changes:
- Complete removal of the "trusted children" thing - now children
processes can be spawned arbitrarily and from arbitrary mountpoint
types, without any additional configuration needed.
- There's a new, required option in the manifest: `libos.entrypoint` - it
specifies the URI to the entry binary in the first process. There's no
need anymore to name the manifest and the first binary identically.
- On SGX, the main binary is not measured in MRENCLAVE anymore - only
PAL, LibOS and the manifest are measured. This is enough to bind
MRENCLAVE to a specific entrypoint user executable if wanted - it
just has to be mounted as a trusted file.
- All Graphene SGX enclaves have now exactly the same MRENCLAVE. This is
a hash of a "Graphene stub", which can "fork" into one of two states
in runtime: initial process or child. The initial process creates a
new "Graphene namespace" with a clean state, it can also be attested
remotely (contrary to child processes). The initial process can spawn
children processes by spawning a Graphene stub and directing it to
start in the child mode. It then attests it locally, and if
successful, establishes an encrypted pipe, "connects" to its own
namespace and treats as trusted (including sending protected files
key).
- Now, there's only one, central manifest describing the initial state
of a Graphene instance which can be spawned from it (previously, each
process required a separate manifest which could have different
configuration - which wasn't actually supported and didn't make sense
design-wise). One downside of central manifests is that all processes
require the same enclave configuration (e.g. size), but that was
already the case so far because of broken checkpointing code. Also,
this is only a temporary problem, which will cease to exist after the
introduction of EDMM.
- `sgx.static_address` was renamed to `sgx.nonpie_binary` and now has to
be inserted manually by users (`sgx_sign` tools doesn't know about the
binaries run inside, which can be even provided or generated in
runtime by the user's workload).
- Caveat: the memory gap for non-PIE executables was removed because it
requires adding a new option to the manifest to be cleanly
implemented. This is left for some future loader rework PR.
This is a major refactor of the way manifests are loaded and handled,
which will be followed by a complete rework of the loader code (which
will include e.g. centralized config).
Changes/fixes:
- Huge part of manifest handling was refactored and untangled.
- Starting without a manifest is now disallowed. This was actually
accidentally broken for some time and no one complained. It also makes
little sense in practice and in Graphene's overall design, e.g. it
conflicts with protected argv.
- Now we only allow starting by giving the executable, not manifest (the
magic resolution logic was removed).
- Now manifests are sent over pipes between parent and children, instead
of children finding and loading them on their own. This is a
preparation for the upcoming centralized manifests change.
- Previously manifests were parsed 2 times on Linux and 3 times on
Linux-SGX (by untrusted PAL, trusted PAL and LibOS). This is now
fixed.
- The common `pal_main()` now requires that the backend-specific PAL
loader loads the manifest before calling it. SGX code already has to
do it (for proper initialization), so let's unify this interface for
all PALs.
- Fix for a PAL crash when manifest size was divisible by page size
(sic!). NULL termination was missing, but most of the time the padding
to page size saved Graphene from crashing.
Pylint output was filtered so that many files with existing pylint
violations were allowed to stay broken.
I made sure all files pass pylint, but whitelisted some rules that
we commonly disable:
* missing docstrings: most of the code is tests/internal anyway
* invalid-name: too many violations, and we commonly use one- or
two-character names (like "a, b" or "t1, t2") which is
disallowed by this rule; we could tweak it and then fix
remaining violations such as camel-case or lowercase constants
* fixme: we leave TODOs as a matter of practice, same as in C
* high-level style rules like too-few-* and too-many-*,
no-self-use
Hopefully that will make using pylint less annoying, while also
catching serious issues (such as unused variables or imports).
Previously, it was possible to use Graphene without the main executable
(e.g. having only preloaded libraries). We aren't aware of anyone using
this weird option, but worse, it led to a very bad UX when the user made
mistakes - e.g. having a typo in the manifest or executable name could
lead to Graphene starting without errors, but doing nothing.
Supporting these options complicates the design of Graphene and loading
logic significantly, providing little useful functionality:
- loader.exec:
- the main user of it were our tests
- worked only for the first process spawned inside Graphene, as it
was a unidirectional manifest->binary mapping, so the child
process didn't know about the corresponding manifest.
- sgx.sigfile:
- probably all existing usages of it were completely redundant
- was resolved relatively to CWD instead of the executable location,
which made it mostly useless
From now on, the correct location of the files is:
- either place the manifest and sigfile next to the binary, with a
matching name, or
- create a symlink to the binary in the folder where manifests are
stored and launch it through this symlink
Running a subset of regression tests, or a single test, is
currently rather annoying, as you cannot simply run 'pytest':
you need to override PYTHONPATH and specify several environment
variables.
Instead, move the whole boilerplate to a wrapper script that
allows running pytest with any command line options. Use it in
Makefiles as well, so that it doesn't go out of sync with CI.
Graphene had some paths to internal files generated at compile time and
hardcoded into the output binary, which disallowed e.g. moving the
Graphene directory after compilation.
Protected files (PF) are a new type of file that can be specified in
the manifest (SGX only). They are encrypted on disk and transparently
decrypted when accessed by the Graphene payload.
Other features:
- data is integrity protected (tamper resistance)
- file swap protection (a PF can only be accessed when in a specific path)
- transparency (Graphene payload sees PFs as regular files, no need to modify
the payload)
See Linux-SGX/protected-files directory for implementation. PF format is
based on protected files from the SGX SDK:
https://github.com/intel/linux-sgx/tree/master/sdk/protected_fs
The following new manifest elements are added:
sgx.protected_files_key = <16-byte hex value>
sgx.protected_files.<name> = file:<host path>
sgx.protected_files_key specifies the encryption key and is only a temporary
implementation. This key should be provisioned with local/remote attestation
in the future.
Paths specifying PF entries can be files or directories. If a directory is
specified, all files/directories within are registered as protected
recursively (and are expected to be encrypted in the PF format).
Linux-SGX/tools directory contains the pf_crypt utility that converts files
to/from the protected format.
Unlike /usr/bin/env which is a Posix standard, there is no guarentee
that /bin/bash exists. This is the case for operating systems such
as FreeBSD, NixOS and Guix. By using /usr/bin/env we also give
the user the option to provide their own bash in a different path
by setting the $PATH environemnt variable.
Distributions usually provide their own packaging wrappers
to fixup shebangs upon installation, however those are not convienent
to use when developing in the source tree.
See also other upstream discussions about the topic:
- https://github.com/systemd/systemd/pull/5816
We don't use it in tests, plus it didn't work on SGX - there was a
warning about autogeneration inserted before the autogenerated shebang.
Also, test_106_manifest_with_shebang didn't actually test the shebang
but ran the binary through the loader, so it was meaningless. We'll need
to fix it and implement again after cleaning up Graphene invocation.
Adapt the python constants so that python-simple also works on
Fedora 31. python-scipy-insecure misses some shared libraries on
Fedora 31, so it does not work there yet.
Adapt Scripts/Makefile.configs so that we can build and test on
Fedora. Most of the tests in Examples are now also runable on
Fedora. Also add a dependency installation target for Fedora to
TensorFlow example.
Extend Makefile.configs and define several variables for make to use
derived from 'gcc -dumpmachine'. In particular:
- ARCH as the architecture, e.g., x86_64
- ARCH_LONG as the long version of the architecture, e.g., x86_64-linux-gnu
- ARCH_LIBDIR as the directory where libraries are located,
e.g., /lib/x86_64-linux-gnu
In Makefiles and manifest templates, replace the hard-coded
x86_64-linux_gnu and /lib/x86_64-linux-gnu through these variables.
Extend the already existing sed scripts to replace the necessary
variables.
The goal of `$(SYS)` check in Makefiles is to skip all targets on
unsupported systems. This commit defines a default goal `all` as
a no-op for simplicity.
Instead of implementing downloading of external resources in every
Makefile again, use one script. This script adds the following
features:
- Always check the download against a known SHA-256 hash.
- Support caching of downloaded resources (set DL_CACHE=/some/dir).
- Allow offline builds if all files are cached. If DL_OFFLINE=true the
build will never attempt to download anything.
Paweł Marczewski