For syscall emulation, use dedicated stack for it.
If application stack is small(e.g. go goroutines), stack overflow happens
and crash. Especially when pushing all the GP registers at the beginning
of syscalldb.
In order to avoid it, allocate dedicated stack area for syscall emulation
and switch them back and forth.
When LibOS emulates unix signal,
- If app is interrupted, setup signal frame directly and return.
- If Pal or LibOS is interrupted, it means app invoked system call and
interrupt. Queue it and check if the signal is queue when LibOS returns to
app and setup signal frame.
- implemented sigreturn and sigaltstack
So LibOS isn't re-entered by signal emulation because app signal handler
isn't triggered while Pal or LibOS is interrupted.
sigreturn needs to restore fp registers.
To avoid interferance to colobber fp registers, make the code path nofp code.
Signed-off-by: Isaku Yamahata <isaku.yamahata@gmail.com>
Previously, Graphene-SGX did not protect send/recv of checkpoint from
parent to child. This leaked all memory contents of the parent process.
This commit adds TLS-PSK (TLS with Pre-Shared Key) wrapper for process
communication. Graphene-SGX already has the logic for SGX-based local
attestation and generation of the shared key for each parent <-> child
communication channel via Diffie–Hellman key exchange. This commit
uses this pre-shared key to create an mbedTLS-based session based on
UNIX domain socketpair (parent.process.stream <-> child.process.stream).
_DkStreamSecure{Init,Free,Read,Write} internal Linux-SGX PAL functions
are added and used during child process creation and sending of the
parent-generated checkpoint. These functions are backed by crypto-layer
lib_SSL{Init,Free,Read,Write} functions which in turn use mbedTLS.
Configuration of mbedTLS is expanded to support TLS-PSK; note that for
entropy source we use only rdrand instruction inside SGX enclave (i.e.,
no untrusted host-platform sources of entropy). The only ciphersuite
currently supported for IPC is MBEDTLS_TLS_PSK_WITH_AES_128_GCM_SHA256.
This commit adds protection only for checkpoint IPC. After the checkpoint
is sent/received, the parent/child processes downgrade their sockets to
plain non-secure ones (by disabling TLS). This is done because child
may spawn a grandchild that also wants to communicate with grandparent,
but it is impossible for multiple processes to share same TLS context.
Additionally, this commit sanitizes "process" PAL handles during send-
handle checkpoint send (via DkSendHandle): shared key and TLS context
are zeroed out.
Previously in Graphene, some getsockopt() syscalls, e.g. TCP_NODELAY,
failed because an underlying PAL handle was't created for the LibOS
handle until bind() was called. Thus, a sequence of accept() and
getsockopt() failed. This commit fixes this by returning default
socket options (possibly augmented with setsockopt values). Test
case is also provided.
Path to enclave file had been a fixed string that is determined at
build time. Therefore Runtime binares were not allowed to move
their location. This commit adds sgx.enclave_pal_file manifest
option to specify the uri of libpal-Linux-SGX.so.
Previously, Graphene failed if recv() contained MSG_PEEK flag. This
resulted in many TLS-based applications failing, including Nginx,
Apache, and Lighttpd in SSL/TLS mode. This commit adds emulation of
MSG_PEEK at LibOS level. A simple TCP test case is provided.
Previously, Graphene incorrectly handled poll/ppoll/select/pselect
of regular and dev files like `/dev/urandom`: it tried to perform
an actual host-OS poll on these files. However, poll of such files
must be emulated completely inside LibOS. This commit adds this
special case to poll/ppoll/select/pselect. A test case is supplied.
Previously, Graphene failed with -EPERM on epoll_ctl(EPOLL_CTL_ADD)
of a LibOS shim handle without a backing PAL handle. However, it is
possible for a LibOS socket handle to not have a PAL handle: if
user app first creates the handle via socket(), then adds it to epoll
via epoll_ctl(), and only then performs bind() at which point PAL
handle is finally created. This commit removes this restriction.
Checkpoint's total memory size is stored in shim_cp_store::mem_size
field. Previously, this field was of type `int`. When a process
allocates more than 2GB of memory and then tries to spawn a child,
the checkpoint send/receive fails due to int overflow of mem_size.
This commit simply changes mem_size type to `size_t`. This is enough
to make the bug go away on e.g. a huge Python app with TensorFlow.
Now Graphene supports an improved version of DkObjectsWaitAny() with
correct polling semantics -- DkObjectsWaitEvents(). This makes
DkObjectsWaitAny() obsolete. This commit removes DkObjectsWaitAny()
and replaces it with:
- DkSynchronizationObjectWait() to wait on a single synchronization
object like mutex or event.
- DkStreamsWaitEvents() to wait on stream-like objects (this is the
renamed DkObjectsWaitEvents()).
The corresponding tests are fixed to use the new PAL interfaces.
Also, IPC helper and Async helper threads are significantly refactored
to make better use of DkStreamsWaitEvents().
This commit adds more cases to the chroot FS to update file metadata:
- update inode number immediately after creating a new file;
- update file size after writing a non-mmapped file.
This commit also adds corresponding LibOS tests.
Syscall recvmsg() accepts an array of buffers (iovecs). This buffers are
filled in array order. POSIX does not allow to fill e.g. iov[0] only
partially before proceeding to iov[1]. This commit makes recvmsg()
emulation compliant with this requirement.
Also, this commit fixes a small performance issue when recvmsg() would
overwrite the same received-address info over and over.
This commit improves the emulation of polling mechanisms (select,
pselect, poll, ppoll, epoll_wait) and cleans up the corresponding
code:
- New DkObjectsWaitEvents() PAL interface, replaces the inefficient
DkObjectsWaitAny() interface. This interface closely resembles
Linux/POSIX poll() in semantics.
- Improved shim_do_epoll_wait() implementation, now using the new
DkObjectsWaitEvents() interface.
- Improved shim_do_poll() implementation, now using the new
DkObjectsWaitEvents() interface.
- Small cleanups of polling code.
Accurate cleanup of shim_do_epoll_create1(), shim_do_epoll_ctl(),
shim_do_epoll_wait(), and other epoll helper functions. This cleanup
also adds error handling (missing previously).
The commit makes (most of) the corresponding LTP tests pass now.
Note that epoll semantics are still incorrect and inefficient: current
epoll_wait() emulation returns only one event to the user.
Accurate cleanup of shim_do_poll(), shim_do_ppoll(), shim_do_select(),
and shim_do_pselect6(). This cleanup also adds error handling (missing
previously).
The commit adds four LibOS regression tests on poll, ppoll, select,
and pselect mechanisms. Also, the corresponding LTP tests pass now.
Eventfd emulation currently relies on the host, thus eventfd syscalls
are disallowed by default due to security concerns. To use them, they
must be explicitly allowed through "sys.allow_insecure_eventfd" in
the manifest.
Isaku Yamahata