150 Commits
Author SHA1 Message Date
Don Porter aede6b6af8 Correct definition of bits per word, as reported in #196. (#217) 2018-10-24 14:53:04 -04:00
Don Porter c68cb42e20 Address a TOCTTOU vulnerability in SGX read/map (#131)
* Fix a bug where configuration error ends up doing a huge allocation, rather than catching the error.  Add some documentation to the slabmgr code.

* Add a unit test and some documentation to answer the question in issue #107.  I can't see how offset and map_start would end up being different.

* Rewrite of SGX file_map to remove TOCTTTOU now passes all unit tests

* Apply a similar fix to file_read.

* Factor complicated verification code into a common helper routine.

* Adjust the memory copying strategy so that all bits in the returned buffed are exactly the same bits as verified in the trusted, scratch buffer.

* Fixing the TOCTOU issue in file checking

* Adding comments for load_trusted_file() and copy_and_check_trusted_file(); Deprecate the old design

* Documenting the file checking mechanism
2018-10-24 14:51:06 -04:00
Don Porter 1dec039d26 Fix race condition of memory mappings during start-up. (#126)
* Fix a system_malloc() race during start-up.  This should address issues #100 and #13
2018-10-24 14:24:15 -04:00
Don Porter 83ab3f9ad1 Error propagation bugfix in PAL initialization (#114)
Fix a bug where configuration error ends up doing a huge allocation, rather than catching the error.  Add some documentation to the slabmgr code.

Fix a stack corruption on signal delivery in a PAL call.  Basically, some important code that saves registers and other state, in case we need to jump out of the PAL, was getting optimized out.  Added some documentation about how the exception handling code works.

A bugfix for getsockname()
2018-10-24 14:08:56 -04:00
Chia-Che Tsai c0e5a761f2 fix a minor bug in ipc/shim_ipc_nsimpl.h 2018-10-24 14:04:58 -04:00
Don Porter a3fcfab658 Fix a bug introduced by commit 3a40af399d (#94)
* Fix a PAL unit test

* Make the PAL unit tests fail properly if one that should work doesn't.

* Get consistent indexes with and without graphene ipc module

* A little debugging output for ltp flakiness.  Seems to make things a little more stable.

* A bugfix for issue #80. The migration is not complete when a file-backed VMA is larger than
the file size, but the file size is not page-aligned. In Linux, if the file size is smaller
than the mapped memory, accessing the remaining memory that is not backed by the file will
trigger a SIGBUS. However, it is fine to access the remaining memory within the last page that
still overlaps with file, and won't trigger a SIGBUS. This area is commonly used in ELF binary
to store uninitialized, static variables. To improve fork latency, Graphene chooses to
truncate the migration size as the file size, but missed the memory which belongs to the last
file-backed page. The migration size should be aligned up to the page size.

* Fix assertion lines in PAL and LibOS

* Make sure the return code is correct for the LTP script, bump up some timeouts

* Only run gipc tests when the module is loaded, for the purposes of getting CI to work.

* Build fix
2018-10-24 14:04:31 -04:00
Chia-Che Tsai d21d75fd7c Bugfixes:
- Fixing AES-CMAC algorithm
- Using SHA512 to hash file stubs (much faster than SHA256 and AES-CMAC)
- Hardening enclave interface (still work-in-progress); delt with issue #28
- Handling socket/pipe polling better
- Allowing setting the lowest heap address in enclaves ('sgx.heap_min' in manifest)
- Fixing the pipe between processes (for SGX)
- Fixing race condition in futex handling in LibOS
- Inheriting epoll handles in forked chilren
- Assigning signal code (now only hard-coding FPE_INTDIV, BUS_ADRERR, SEGV_ACCERR, SEGV_MAPERR)
- Fixing race condition in vma allocation (likely to fix bug() in bookkeep/shim_vma.c)
- Clearer debug message in LibOS
- Fixing calling convention in LibOS and glibc
- Fixing futex behavior (FUTEX_WAIT takes relative time, FUTEX_WAIT_BITSET takes absolute time)
- More system call implemented
- More application working (NGINX)
2018-10-24 13:49:21 -04:00
Chia-Che Tsai 4f1d69c384 fix for large files within an SGX enclave (#136)
Replacing some "int" in the code base as "int64_t" or "uint64_t".
2018-10-24 11:57:09 -04:00
Chia-Che Tsai 776165c811 Lots of bugfixes (#145)
- Fix a severe bug caused by merging #87. The patch does not check the return values from get_config_entries_size(), while the return values can easily be negative (-PAL_ERROR_INVAL). Any usage of this function should carefully check the return values and also pass the size into get_config_entries() to prevent buffer overflow.
- Fix a minor bug in handling IPC disconnection. The callback ipc_child_exit() never receives a "term_signal" parameter. The callback is called when thepipe to a child process unexpectedly terminates, which can only mean the child process has crashed. I believe the proper signal to send is SIGKILL.
- Remove a compilation warning caused by type mismatch of __malloc().
- Allow binding TCP socket to ANY address
- enable LD_PRELOAD in glibc
- disable output buffering in LTP tests to reveal omitted passes
- Remove a double-unlock in shim_async.c
- Replace all int with size_t or ssize_t in Graphene configuration API
- Add new passed LTP tests
2018-10-24 11:56:24 -04:00
Don Porter 4997ca93b9 CI Improvements (#141)
Run CI in a Docker image

* Wait a second for lighttp and apache to fully initialize the socket

* Try to add a timeout for lmbench, which is prone to hanging (vfork)

* Set a timeout on gcc; one of the PRs is causing gcc hello to hang.  Also, to avoid interference, run apache test on 8001 by default.

* Update lmbench to ignore bin dir

* Do the faster tests first; fail earlier

* Get the IP addr properly for web server tests

* Some notes on flaky LTP tests to debug later.  Move a few that tend to heisenbugs off of the required list for now

* Try to detect vma list corruption during the debug dump, lest it be infinite.

* Temporarily disable the lmbench vfork test (Issue #142).

* Catch some failures faster.

* Catch failures in apachebench

* Take apache out of the Linux host/Debug build CI for now, as it hangs consistently.  Filed issue #144 to track.
2018-08-13 13:29:43 -04:00
Chia-Che Tsai 3dd1d00eb6 Fixing a bunch of memory leaking issues (#86)
* add memusg to pal_loader script

* Stop bleeding PAL handles.

Deprecating DkOjectReference and reference counting in PAL handle.
Deprecating DkSemaphoreDestroy and DkEventDestroy (replaced by DkObjectClose).
Cleaning unused PAL handles in the library OS.
Adding a heap tracing feature to profile usage of PAL handles.

* fix a bug in SGX mode that mapping untrusted files into memory never got free by DkVirtualMemoryFree()

* adding lighttpd SSL option

* fixing the freeing convention of PAL handles; On SGX, event and mutex handles need to be freed seperately.

* changing how mutexes and events are allocated on SGX

* fixing GCC regression tests (for both Linux and SGX)

* fix a double-free problem of the first thread handle
2018-08-13 13:28:05 -04:00
Don Porter e84d9ac302 Move regression script down one level 2018-08-13 13:20:25 -04:00
Chia-Che Tsai aed2a52164 adding rules to build manifests in LibOS/shim/test/regression 2018-08-13 13:20:13 -04:00
Don Porter 43c83914f1 This unit test needs a manifest template 2018-08-13 13:20:01 -04:00
Chia-Che Tsai ee0f4f5e3b A bugfix for issue #80 (os.fork() sigfaults in Python) (#81)
* A bugfix for issue #80. The migration is not complete when a file-backed VMA is larger than
the file size, but the file size is not page-aligned. In Linux, if the file size is smaller
than the mapped memory, accessing the remaining memory that is not backed by the file will
trigger a SIGBUS. However, it is fine to access the remaining memory within the last page that
still overlaps with file, and won't trigger a SIGBUS. This area is commonly used in ELF binary
to store uninitialized, static variables. To improve fork latency, Graphene chooses to
truncate the migration size as the file size, but missed the memory which belongs to the last
file-backed page. The migration size should be aligned up to the page size.

* adding several test cases for mmap with files

* adding /tmp to the ltp manifest

* add documentation about the corner cases of mmap + fork
2018-08-13 13:19:51 -04:00
Don Porter 123476ede5 Migrate unit tests to jenkins (#122)
* Migrate running unit tests for Linux host to Jenkins.  Add a unit test for the Linux host, Debug build

* Ignore files generated as part of unit testsing.

* A quick and dirty fix for port collisions during CI tests.  Come back and do a better job if we continue having problems

* For some reason, the PAL Process regression tests fail using the default manifest under a debug build.  Go ahead and explicate a template for the Process test.
2018-08-13 10:31:01 -04:00
Don Porter e2c8df8038 More CI debugging 2018-08-13 10:30:24 -04:00
Don Porter 89485a4350 More timeouts. 2018-08-13 10:30:00 -04:00
Don Porter a19f2b1eff More CI debugging 2018-08-13 10:29:50 -04:00
Don Porter 211bab0a36 More CI tweaking 2018-08-13 10:29:36 -04:00
Jaehyun Han da82acd9e4 Fix the limitation on the number of trusted files (#87)
* Support a large number of trusted files. (Issue #9)

* Config counts overall key size of entries
2018-08-13 10:29:27 -04:00
Don Porter 78ea16ab53 More timeout tweaking 2018-08-13 10:26:57 -04:00
Don Porter d7d37b64e8 More timeout and gitignore tweaks 2018-08-13 10:26:31 -04:00
Don Porter be44dc84f2 More notes on license 2018-08-13 10:24:39 -04:00
Don Porter 3777b0671f One more timeout 2018-08-13 10:22:07 -04:00
Don Porter 662c253edf One more timeouts 2018-08-13 10:21:40 -04:00
Don Porter e169ee698f More CI debugging 2018-08-13 10:21:32 -04:00
Don Porter 1a63a16deb More CI debugging 2018-08-13 10:20:49 -04:00
Don Porter 1a46e2fbe2 More CI debugging 2018-08-13 10:20:18 -04:00
Don Porter 2974af0252 More timeouts 2018-08-13 10:20:05 -04:00
Don Porter 62b57a4f79 Bump up a few LTP timeouts 2018-08-13 10:20:00 -04:00
Don Porter fe7a6e6934 Fix a bug introduced by commit 3a40af399d (#94)
* Fix a PAL unit test

* Make the PAL unit tests fail properly if one that should work doesn't.

* Get consistent indexes with and without graphene ipc module

* A little debugging output for ltp flakiness.  Seems to make things a little more stable.

* A bugfix for issue #80. The migration is not complete when a file-backed VMA is larger than
the file size, but the file size is not page-aligned. In Linux, if the file size is smaller
than the mapped memory, accessing the remaining memory that is not backed by the file will
trigger a SIGBUS. However, it is fine to access the remaining memory within the last page that
still overlaps with file, and won't trigger a SIGBUS. This area is commonly used in ELF binary
to store uninitialized, static variables. To improve fork latency, Graphene chooses to
truncate the migration size as the file size, but missed the memory which belongs to the last
file-backed page. The migration size should be aligned up to the page size.

* Fix assertion lines in PAL and LibOS

* Make sure the return code is correct for the LTP script, bump up some timeouts

* Only run gipc tests when the module is loaded, for the purposes of getting CI to work.

* Build fix
2018-08-13 10:19:44 -04:00
Chia-Che Tsai c187d67e31 add a python test script 2018-08-13 10:18:05 -04:00
Chia-Che Tsai 929e89fe95 update regression scripts 2018-08-13 10:17:59 -04:00
Chia-Che Tsai bb61416ff7 update regression scripts 2018-08-13 10:17:55 -04:00
Chia-Che Tsai ff7a4b9a4f update openjdk and python scripts 2018-08-13 10:17:44 -04:00
Chia-Che Tsai 9a300fa06c update openjdk and python scripts 2018-08-13 10:17:39 -04:00
Chia-Che Tsai e271d2c95e reverse changes to the OpenJDK 7 directory 2018-08-13 10:17:26 -04:00
Don Porter 77d5e57563 Fix a few library versions for the latest 16.04. We probably want a better way to handle this across systems. 2018-08-13 10:16:30 -04:00
Don Porter 81c78dde43 Drop heap_min. 2018-08-13 10:15:37 -04:00
Chia-Che Tsai 629e30a2d2 Add "make regression" rule to the Python directory 2018-08-13 10:14:14 -04:00
Chia-Che Tsai 73c484214e fix the problem that hello.token is not created whe running make SGX_RUN=1'' in the lmbench directory 2018-08-13 10:14:10 -04:00
Don Porter b7ce0a293c Link the script for apache 2018-08-13 10:13:57 -04:00
Don Porter 31c364b757 Save the web server benchmarking script for regression testing. 2018-08-13 10:13:52 -04:00
Chia-Che Tsai f8231317f0 fix the building scripts of apache 2018-08-13 10:11:25 -04:00
Chia-Che Tsai dc0644518b add Graphene-SGX building and non-SGX regression tests into .travis.yml 2018-08-13 10:11:19 -04:00
Chia-Che Tsai 92d949c11f a minor typo in GCC's manifest 2018-08-13 10:11:11 -04:00
Chia-Che Tsai 2a6171518e a trivial change in LTP Makefile 2018-08-13 10:11:06 -04:00
Chia-Che Tsai b4936dd531 Adding all the LTP tests recently passed 2018-08-13 10:11:00 -04:00
Chia-Che Tsai 69c1dda01e Add "regression" rules for LMBench and GCC 2018-08-13 10:10:54 -04:00