FreeBSD PAL is not maintained anymore, is neither built nor tested in
our CI and most likely stopped working long time ago. Currently it only
confuses people implementing changes in PALs (should they also fix bugs
in FreeBSD PAL if they can't test it?) and gives a false impression that
we support FreeBSD hosts.
Syscall recvmsg() accepts an array of buffers (iovecs). This buffers are
filled in array order. POSIX does not allow to fill e.g. iov[0] only
partially before proceeding to iov[1]. This commit makes recvmsg()
emulation compliant with this requirement.
Also, this commit fixes a small performance issue when recvmsg() would
overwrite the same received-address info over and over.
Jenkins should complain about new files and those already fixed. After
fixing a file completely, the file path should be removed from the list.
This is helpful for pylint, because some errors, mostly from category
"refactor", are attributed to the line with function's signature and are
liable to be missed by the heuristics.
Fixes warnings in: pal-sgx-sign, pal-sgx-get-token and pal-gdb.py.
Ignores them in aesm_pb2.py and Documentation/conf.py. The former is
autogenerated, the latter is mostly irrelevant. If someone starts
messing with it, we could re-enable this, but Sphinx' requirements clash
with pylint, mostly on invalid- name, because the configuration
variables are expected lower-case.
PAL_TCB_LINUX of Pal/Linux-SGX is unrelated to PAL_TCB because untrusted
code is independent from trusted code. Remove PAL_TCB from PAL_TCB_LINUX
and make PAL_TCB_LINUX visible only to untrusted code. Also, rename to
PAL_TCB_URTS to make it clear that it belongs to untrusted code.
This commit improves the emulation of polling mechanisms (select,
pselect, poll, ppoll, epoll_wait) and cleans up the corresponding
code:
- New DkObjectsWaitEvents() PAL interface, replaces the inefficient
DkObjectsWaitAny() interface. This interface closely resembles
Linux/POSIX poll() in semantics.
- Improved shim_do_epoll_wait() implementation, now using the new
DkObjectsWaitEvents() interface.
- Improved shim_do_poll() implementation, now using the new
DkObjectsWaitEvents() interface.
- Small cleanups of polling code.
Accurate cleanup of shim_do_epoll_create1(), shim_do_epoll_ctl(),
shim_do_epoll_wait(), and other epoll helper functions. This cleanup
also adds error handling (missing previously).
The commit makes (most of) the corresponding LTP tests pass now.
Note that epoll semantics are still incorrect and inefficient: current
epoll_wait() emulation returns only one event to the user.
Accurate cleanup of shim_do_poll(), shim_do_ppoll(), shim_do_select(),
and shim_do_pselect6(). This cleanup also adds error handling (missing
previously).
The commit adds four LibOS regression tests on poll, ppoll, select,
and pselect mechanisms. Also, the corresponding LTP tests pass now.
This commit refactors db_object.c of Linux and Linux-SGX PALs:
- Remove _DkObjectWaitOne() and incorporate its special-case of
waiting on single mutex/event in _DkObjectsWaitAny().
- Refactor _DkObjectsWaitAny() for readability.
- Remove unused DEFAULT_QUANTUM and TRACE_HEAP_LEAK.
This commit adds `sgx.ra_accept_configuration_needed` manifest option
to handle the case when SGX remote attestation from IAS returns status
CONFIGURATION_NEEDED (which means that this SGX platform is not known
to be compromised but has e.g. hyper-threading enabled).
Eventfd emulation currently relies on the host, thus eventfd syscalls
are disallowed by default due to security concerns. To use them, they
must be explicitly allowed through "sys.allow_insecure_eventfd" in
the manifest.
Previously, parent and child processes communicated via two unidirectional
pipes: process.stream_in and process.stream_out. Now these pipes were replaced
with bidirectional socketpairs, so there is no need to distinguish between
read and write ends. This commit merges stream_in and steam_out FDs into one
stream FD, and simplifies corresponding code (especially SGX implementation).
Graphene now has a reworked parent-child verification and SGX local
attestation protocol. Remove the dead commented-out code that is not
needed anymore, along with unused macros.
Graphene already emulates pipes via UNIX domain socketpairs. The only
place where Graphene still uses host pipes is in DkCreateProcess(),
for communication and checkpoint send/receive between parent and child.
UNIX domain socketpairs are more convenient than pipes and allow
bidirectional communication, which is useful for IPC encryption via
SSL/TLS. This commit replaces all lingering uses of pipes with
socketpairs.
Graphene IPC (GIPC) was introduced to perform faster bulk IPC by sharing pages as copy-on-write
across processes. This feature became stale, and it was shown that recent Linux kernels (4.2+)
have zero-copy transfers over UNIX sockets and exhibit similar performance. GIPC is not built and
not tested in Jenkins. Also, GIPC does not work under SGX. This commit completely removes GIPC.
Test classes and functions are named TC_01_Example and test_012_example,
respectively. Those are not conforming to the usual style enforced by
pylint but that's okay.
This commit completely rewrites futex implementation to (hopefully)
remove all races (both on memory access level and between waits and
wakes), make it compatible with actual Linux implementation and make
it more maintainable and readable.
Previously, there was a segfault if a child enclave issued a cpuid instruction
during init_child_process(), because the enclave was not yet set up to handle
exceptions. This commit moves this set up earlier in initialization sequence.
Previously, if user performed getdents() on a directory containing
inaccessible files (because user doesn't have permission), whole
getdents failed with -EACCES. This is incorrect behavior: files must
still be listed. This commit fixes the root cause of this bug by
marking inaccessible files as DENTRY_NEGATIVE.
Previously, exit_group test was disabled on Linux-SGX PAL. It was
disabled because the PAL incorrectly exited threads. The latest
commits fix this, and the test can be re-enabled.
Previously, there was a data race on thread::is_alive between one thread
checking whether it is the last thread alive via check_last_thread() and
another thread exiting via thread_exit(). The former checks if is_alive
is true, the latter sets it to false. However, the exiting thread will
truly exit only after it called DkThreadExit(), thus the race on is_alive
led to scenarios where two threads believe to be the last threads alive
and compete on terminating Async Helper/IPC threads and exiting the whole
process. This commit introduces cleanup_thread() called by Async Helper
to set is_alive to false and delete the thread, freeing its resources.
The data race is thus removed, and shim_thread object leak is prevented.
In Linux-SGX PAL, the main thread's stack is provided by host Linux, unlike
child threads which stacks are mmapped by the PAL. Therefore, the main
thread's stack must not be munmapped, unlike child threads' stacks.
Previously, PAL tried to munmap stack of even the main thread, leading to
spurious segfaults in e.g. `abort_multithread` test.
Previously, it took more than 30-45 minutes to run one SGX pipeline.
This commit splits that SGX pipeline into two: one runs LibOS/PAL
tests and LTP tests, the other one runs only application examples.
The times to run each of these pipelines are roughly the same.
Splitting SGX pipelines decreases wait time for Jenkins in half.
The pipeline is failed only when pylint whines about the code actually
touched as part of the pull request.
The `run-shellcheck` script is provided so it can be reused, maybe in
`.git/hooks/pre-push`.
The pipeline is failed only when pylint whines about the code actually
touched as part of the pull request.
Ubuntu 16.04 pipelines are not enabled, since they are affected by this
bug, since fixed: https://github.com/PyCQA/pylint/issues/1216.
The `run-pylint` script is provided so it can be reused, maybe in
`.git/hooks/pre-push`.
Michał Kowalczyk