It turns out that our guard pages were incorrectly handled (i.e. not re-
added to LibOS VMA list) on SGX when execve was optimized to re-use the
same enclave, which caused exec_same test to crash from time to time
(when ASLR put heap on a guard page).
Static guard pages aren't too useful and introduce unnecessary
complexity to our code, so we decided to just delete them in order to
fix this bug.
This commit adds the ability to provision the wrap (master) key for
protected files at runtime (in contrast to previous approach of
hard-coding `protected_files_key` in the manifest as a temporary
solution). This is achieved as follows:
- New PAL API `DkSetProtectedFilesKey()` is added.
- New writable pseudo-file `/dev/attestation/protected_files_key` is
added. It calls `DkSetProtectedFilesKey()` after it was written to.
- New `SECRET_PROVISION_SET_PF_KEY` option is added to the Secret
Provisioning library. If it is set, the library assumes that the
first provisioned secret is the wrap key for PF and writes it into
the new pseudo-file.
The Secret Provisioning example `ra-tls-secret-prov` is updated to
include the new protected-files client. This client receives the wrap
key for PF via secret provisioning and reads & outputs the protected
file `files/input.txt`.
*NOTE*: The current implementation of provisioning the wrap key does
not work for `loader.argv_src_file` and `loader.env_src_file` if they
point to protected files (because provisioning happens after setting
up arguments and environment variables).
This patch optimizes _DkSystemTimeQuery() using RDTSC instead of
ocall_gettime.
This optimization won't take effect if there is no reliable TSC source
available to use i.e. nonstop/invariant TSC.
The TSC drift is bound by syncing with system clock periodically.
Protected files (PF) are a new type of file that can be specified in
the manifest (SGX only). They are encrypted on disk and transparently
decrypted when accessed by the Graphene payload.
Other features:
- data is integrity protected (tamper resistance)
- file swap protection (a PF can only be accessed when in a specific path)
- transparency (Graphene payload sees PFs as regular files, no need to modify
the payload)
See Linux-SGX/protected-files directory for implementation. PF format is
based on protected files from the SGX SDK:
https://github.com/intel/linux-sgx/tree/master/sdk/protected_fs
The following new manifest elements are added:
sgx.protected_files_key = <16-byte hex value>
sgx.protected_files.<name> = file:<host path>
sgx.protected_files_key specifies the encryption key and is only a temporary
implementation. This key should be provisioned with local/remote attestation
in the future.
Paths specifying PF entries can be files or directories. If a directory is
specified, all files/directories within are registered as protected
recursively (and are expected to be encrypted in the PF format).
Linux-SGX/tools directory contains the pf_crypt utility that converts files
to/from the protected format.
Graphene used its own implementations of memory-handling functions like
memcmp(), memcpy(), etc. These implementations were copied from Glibc
and contained complicated code from year 2004. Modern HW and compilers
do better job at optimizing simple C implementations of these functions.
Thus, this commit replaces old implementations with simple ones taken
from Musl libc and adapted to our code style.
One particular optimization is made to memcpy() on x86-64. memcpy() is
heavily used in Linux-SGX PAL to copy data in/out of SGX enclave.
Experiments with Redis 5.0 show perf improvement of using "rep movsb" at
3-5% for 4KB payloads over the previous implementation based on Glibc.
This is the first step in removing this obsolete header.
Additionally, AtomicMath test is removed, as it became obsolete after
these changes (and was relying on undefined behaviors anyway).
Previously, Graphene always performed ASLR at the LibOS layer. ASLR
may lead to a situation when one mmap allocates an object in the
middle of address space, and there is no space for a later mmap of
a large object. This is problematic in restricted environments such
as SGX enclaves. In particular, `large_mmap` LibOS test failed
occasionally because it only has 8GB of enclave size and it may
mmap first objects somewhere in the middle (around 4GB address) and
then fail to find any space for a large 4GB mmap.
This commit adds "loader.insecure__disable_aslr" manifest option.
If it set to one, ASLR is disabled and mappings become deterministic
which guarantees programs like `large_mmap` never fail due to ENOMEM.
Previously, Graphene simply forwarded SIGPIPE generated by the host to
LibOS/app. Unfortunately, SIGPIPE generation is a process-wide feature
and there is no portable way to restrict it only to a subset of pipes,
UNIX domain sockets, etc. This led to sporadic Graphene failures
because Graphene's internal use of pipes and sockets may result in an
unexpected (to application) SIGPIPE.
This commit removes the forwarding of SIGPIPE. Instead, PALs explicitly
ignore SIGPIPE. This forces the host to return EPIPE error code, which
is checked only on a subset of LibOS handles (the ones created by the
app), and if required, LibOS generates a SIGPIPE for the application.
While adding this logic, the whole PAL exception code was refactored,
both in Linux and Linux-SGX. Tests for SIGPIPE are now enabled for
both Linux and Linux-SGX PALs.
Use the ucontext from PAL instead. LibOS now has access to the inline
functions for copying PAL_CONTEXT to ucontext and vice versa and we use
them where possible.
We need to introduce a ucontext.h for Skeleton. It does need ucontext
to be defined for being able to compile shim_signal.c. The easiest way
to achieve this is to rely on Linux's ucontext.h.
SGX can reuse Linux's ucontext.h and sigcontext.h.
Move the Linux x86_64 specific syscall arch_prctrl into a new inline function
pal_set_tcb located in include/arch/x86_64/Linux/pal_host-arch.h. The SGX and
Skeleton builds now also need a pal_host-arch.h file, empty for now.
Implement inline functions for copying CPU context between PAL_CONTEXT
and ucontext_t. Add an assert to make sure that the number of registers
in both contexts is the same.
This patch adds a new PAL_EVENT_PIPE to handle EPIPE signals
and forward them to the signal handler installed in LibOS.
If the application does not have a signal handler installed,
it will terminate the application with SIGPIPE exit code.
Support for SIGPIPE in Linux-SGX PAL will be added in a follow-up
commit.
Also, adapt #includes where needed. Avoid the name elf.h to avoid
clashes. We do not touch the Linux-SGX/elf-x86_64.h file since it is
slightly different.
Introduce PAL_ERROR_CONNFAILED_PIPE and treat EPIPE separately
from ECONNRESET.
The effects of this patch on LTP are:
from:
writev01.c:139: FAIL: write to closed pipe, expected: -1 (EPIPE), got: -1 (ECONNRESET)
to:
writev01.c:139: PASS: write to closed pipe, expected: -1 (EPIPE), got: -1 (EPIPE)
AND:
from:
write05.c:82: FAIL: write() failed unexpectedly, expected EPIPE: ECONNRESET
to:
write05.c:87: FAIL: sigpipe_cnt = 0
writev01 now works correctly, so this commit enables it.
Move the x86-64-specific sigcontext header files to arch/x86_64/Linux.
The SGX and non-SGX files are identical.
We are also moving sigset.h since on ppc64 the following defines are
different:
x86_64: #define _SIGSET_NWORDS (64 / (8 * sizeof(unsigned long int)))
ppc64: #define _SIGSET_NWORDS (1024 / (8 * sizeof (unsigned long int)))
Also, adapt the Makefiles to add the arch specific directory to the CFLAGS.
The Linux-SGX sysdep-x86_64.h was identical and could therefore be removed.
Sometimes we need to prevent the compiler from reading or writing to
a memory location twice to prevent certain TOCTOU bugs. This can now
be achieved by using the introduced macros and this commit does so in
enclave_ocalls.c for Linux-SGX.
This commit completely reworks VMA subsystem along with its usages.
New version should be: cleaner (easier to maintain), faster and allow
for bookkeeping requests from Pal.
It also fixes some bugs and inconsistencies found in the process and
changes brk and mmap/munmap implementations (at least partially).
Currently various flags in file and memory syscalls work mostly by an
accident, because values of some of them align with corresponding Linux
syscall flags. Some APIs weren't that lucky though - e.g.
DkStreamOpen(..., /*options=*/PAL_OPTION_CLOEXEC) deletes file contents
(sic!) intead of opening it with O_CLOEXEC. This is because
PAL_OPTION_CLOEXEC == O_TRUNC.
This commit fixes all this mess and also adds asserts to check validity
of flags passed to Dk* handlers.
mbedTLS configuration used in Graphene is not thread-safe (because
this would require the use of a threading library like pthread which
is not possible in the LibOS/Pal layers). However, some mbedTLS
functions use shared state, in particular TLS context initialization
functions. This led to data races during encrypted-pipe creation,
since it requires two threads performing a TLS handshake. This commit
refactors TLS init into SSLInit (not thread-safe) and SSLHandshake
(thread-safe) and adds spinlocks around SSLInit to protect the racy
mbedTLS logic.
Michał Kowalczyk