234 Commits
Author SHA1 Message Date
Don Porter aede6b6af8 Correct definition of bits per word, as reported in #196. (#217) 2018-10-24 14:53:04 -04:00
Don Porter c68cb42e20 Address a TOCTTOU vulnerability in SGX read/map (#131)
* Fix a bug where configuration error ends up doing a huge allocation, rather than catching the error.  Add some documentation to the slabmgr code.

* Add a unit test and some documentation to answer the question in issue #107.  I can't see how offset and map_start would end up being different.

* Rewrite of SGX file_map to remove TOCTTTOU now passes all unit tests

* Apply a similar fix to file_read.

* Factor complicated verification code into a common helper routine.

* Adjust the memory copying strategy so that all bits in the returned buffed are exactly the same bits as verified in the trusted, scratch buffer.

* Fixing the TOCTOU issue in file checking

* Adding comments for load_trusted_file() and copy_and_check_trusted_file(); Deprecate the old design

* Documenting the file checking mechanism
2018-10-24 14:51:06 -04:00
Don Porter 1dec039d26 Fix race condition of memory mappings during start-up. (#126)
* Fix a system_malloc() race during start-up.  This should address issues #100 and #13
2018-10-24 14:24:15 -04:00
Don Porter 83ab3f9ad1 Error propagation bugfix in PAL initialization (#114)
Fix a bug where configuration error ends up doing a huge allocation, rather than catching the error.  Add some documentation to the slabmgr code.

Fix a stack corruption on signal delivery in a PAL call.  Basically, some important code that saves registers and other state, in case we need to jump out of the PAL, was getting optimized out.  Added some documentation about how the exception handling code works.

A bugfix for getsockname()
2018-10-24 14:08:56 -04:00
Chia-Che Tsai c0e5a761f2 fix a minor bug in ipc/shim_ipc_nsimpl.h 2018-10-24 14:04:58 -04:00
Don Porter a3fcfab658 Fix a bug introduced by commit 3a40af399d (#94)
* Fix a PAL unit test

* Make the PAL unit tests fail properly if one that should work doesn't.

* Get consistent indexes with and without graphene ipc module

* A little debugging output for ltp flakiness.  Seems to make things a little more stable.

* A bugfix for issue #80. The migration is not complete when a file-backed VMA is larger than
the file size, but the file size is not page-aligned. In Linux, if the file size is smaller
than the mapped memory, accessing the remaining memory that is not backed by the file will
trigger a SIGBUS. However, it is fine to access the remaining memory within the last page that
still overlaps with file, and won't trigger a SIGBUS. This area is commonly used in ELF binary
to store uninitialized, static variables. To improve fork latency, Graphene chooses to
truncate the migration size as the file size, but missed the memory which belongs to the last
file-backed page. The migration size should be aligned up to the page size.

* Fix assertion lines in PAL and LibOS

* Make sure the return code is correct for the LTP script, bump up some timeouts

* Only run gipc tests when the module is loaded, for the purposes of getting CI to work.

* Build fix
2018-10-24 14:04:31 -04:00
Chia-Che Tsai d21d75fd7c Bugfixes:
- Fixing AES-CMAC algorithm
- Using SHA512 to hash file stubs (much faster than SHA256 and AES-CMAC)
- Hardening enclave interface (still work-in-progress); delt with issue #28
- Handling socket/pipe polling better
- Allowing setting the lowest heap address in enclaves ('sgx.heap_min' in manifest)
- Fixing the pipe between processes (for SGX)
- Fixing race condition in futex handling in LibOS
- Inheriting epoll handles in forked chilren
- Assigning signal code (now only hard-coding FPE_INTDIV, BUS_ADRERR, SEGV_ACCERR, SEGV_MAPERR)
- Fixing race condition in vma allocation (likely to fix bug() in bookkeep/shim_vma.c)
- Clearer debug message in LibOS
- Fixing calling convention in LibOS and glibc
- Fixing futex behavior (FUTEX_WAIT takes relative time, FUTEX_WAIT_BITSET takes absolute time)
- More system call implemented
- More application working (NGINX)
2018-10-24 13:49:21 -04:00
Chia-Che Tsai 4f1d69c384 fix for large files within an SGX enclave (#136)
Replacing some "int" in the code base as "int64_t" or "uint64_t".
2018-10-24 11:57:09 -04:00
Chia-Che Tsai 776165c811 Lots of bugfixes (#145)
- Fix a severe bug caused by merging #87. The patch does not check the return values from get_config_entries_size(), while the return values can easily be negative (-PAL_ERROR_INVAL). Any usage of this function should carefully check the return values and also pass the size into get_config_entries() to prevent buffer overflow.
- Fix a minor bug in handling IPC disconnection. The callback ipc_child_exit() never receives a "term_signal" parameter. The callback is called when thepipe to a child process unexpectedly terminates, which can only mean the child process has crashed. I believe the proper signal to send is SIGKILL.
- Remove a compilation warning caused by type mismatch of __malloc().
- Allow binding TCP socket to ANY address
- enable LD_PRELOAD in glibc
- disable output buffering in LTP tests to reveal omitted passes
- Remove a double-unlock in shim_async.c
- Replace all int with size_t or ssize_t in Graphene configuration API
- Add new passed LTP tests
2018-10-24 11:56:24 -04:00
Chia-Che Tsai 857e92b03f Fixing a bunch of memory leaking issues (#86)
* add memusg to pal_loader script

* Stop bleeding PAL handles.

Deprecating DkOjectReference and reference counting in PAL handle.
Deprecating DkSemaphoreDestroy and DkEventDestroy (replaced by DkObjectClose).
Cleaning unused PAL handles in the library OS.
Adding a heap tracing feature to profile usage of PAL handles.

* fix a bug in SGX mode that mapping untrusted files into memory never got free by DkVirtualMemoryFree()

* adding lighttpd SSL option

* fixing the freeing convention of PAL handles; On SGX, event and mutex handles need to be freed seperately.

* changing how mutexes and events are allocated on SGX

* fixing GCC regression tests (for both Linux and SGX)

* fix a double-free problem of the first thread handle
2018-10-24 11:31:19 -04:00
Don Porter 4997ca93b9 CI Improvements (#141)
Run CI in a Docker image

* Wait a second for lighttp and apache to fully initialize the socket

* Try to add a timeout for lmbench, which is prone to hanging (vfork)

* Set a timeout on gcc; one of the PRs is causing gcc hello to hang.  Also, to avoid interference, run apache test on 8001 by default.

* Update lmbench to ignore bin dir

* Do the faster tests first; fail earlier

* Get the IP addr properly for web server tests

* Some notes on flaky LTP tests to debug later.  Move a few that tend to heisenbugs off of the required list for now

* Try to detect vma list corruption during the debug dump, lest it be infinite.

* Temporarily disable the lmbench vfork test (Issue #142).

* Catch some failures faster.

* Catch failures in apachebench

* Take apache out of the Linux host/Debug build CI for now, as it hangs consistently.  Filed issue #144 to track.
2018-08-13 13:29:43 -04:00
Chia-Che Tsai 9be6b65d2a update README about submodules and clarify about custom kernel usage 2018-08-13 13:28:17 -04:00
Chia-Che Tsai 3dd1d00eb6 Fixing a bunch of memory leaking issues (#86)
* add memusg to pal_loader script

* Stop bleeding PAL handles.

Deprecating DkOjectReference and reference counting in PAL handle.
Deprecating DkSemaphoreDestroy and DkEventDestroy (replaced by DkObjectClose).
Cleaning unused PAL handles in the library OS.
Adding a heap tracing feature to profile usage of PAL handles.

* fix a bug in SGX mode that mapping untrusted files into memory never got free by DkVirtualMemoryFree()

* adding lighttpd SSL option

* fixing the freeing convention of PAL handles; On SGX, event and mutex handles need to be freed seperately.

* changing how mutexes and events are allocated on SGX

* fixing GCC regression tests (for both Linux and SGX)

* fix a double-free problem of the first thread handle
2018-08-13 13:28:05 -04:00
Don Porter e84d9ac302 Move regression script down one level 2018-08-13 13:20:25 -04:00
Chia-Che Tsai aed2a52164 adding rules to build manifests in LibOS/shim/test/regression 2018-08-13 13:20:13 -04:00
Don Porter 43c83914f1 This unit test needs a manifest template 2018-08-13 13:20:01 -04:00
Chia-Che Tsai ee0f4f5e3b A bugfix for issue #80 (os.fork() sigfaults in Python) (#81)
* A bugfix for issue #80. The migration is not complete when a file-backed VMA is larger than
the file size, but the file size is not page-aligned. In Linux, if the file size is smaller
than the mapped memory, accessing the remaining memory that is not backed by the file will
trigger a SIGBUS. However, it is fine to access the remaining memory within the last page that
still overlaps with file, and won't trigger a SIGBUS. This area is commonly used in ELF binary
to store uninitialized, static variables. To improve fork latency, Graphene chooses to
truncate the migration size as the file size, but missed the memory which belongs to the last
file-backed page. The migration size should be aligned up to the page size.

* adding several test cases for mmap with files

* adding /tmp to the ltp manifest

* add documentation about the corner cases of mmap + fork
2018-08-13 13:19:51 -04:00
Don Porter 123476ede5 Migrate unit tests to jenkins (#122)
* Migrate running unit tests for Linux host to Jenkins.  Add a unit test for the Linux host, Debug build

* Ignore files generated as part of unit testsing.

* A quick and dirty fix for port collisions during CI tests.  Come back and do a better job if we continue having problems

* For some reason, the PAL Process regression tests fail using the default manifest under a debug build.  Go ahead and explicate a template for the Process test.
2018-08-13 10:31:01 -04:00
Don Porter c8571258e3 Update my email 2018-08-13 10:30:30 -04:00
Don Porter e2c8df8038 More CI debugging 2018-08-13 10:30:24 -04:00
Neil Davis ddc85d1b7f Removed DEBUG=1 (#121)
* Removed DEBUG=1 from basic Jenkins build/test process.

* Removed the 'input' so that the Jenkinsfile finishes when necessary
2018-08-13 10:30:16 -04:00
Don Porter 89485a4350 More timeouts. 2018-08-13 10:30:00 -04:00
Don Porter a19f2b1eff More CI debugging 2018-08-13 10:29:50 -04:00
Don Porter 2265e28686 More CI debugging. I think we should run the non-SGX unit tests before alternate builds. 2018-08-13 10:29:44 -04:00
Don Porter 211bab0a36 More CI tweaking 2018-08-13 10:29:36 -04:00
Jaehyun Han da82acd9e4 Fix the limitation on the number of trusted files (#87)
* Support a large number of trusted files. (Issue #9)

* Config counts overall key size of entries
2018-08-13 10:29:27 -04:00
Don Porter 78ea16ab53 More timeout tweaking 2018-08-13 10:26:57 -04:00
Don Porter d7d37b64e8 More timeout and gitignore tweaks 2018-08-13 10:26:31 -04:00
Don Porter 7d68605137 More license notes 2018-08-13 10:26:04 -04:00
Don Porter be44dc84f2 More notes on license 2018-08-13 10:24:39 -04:00
Don Porter 3777b0671f One more timeout 2018-08-13 10:22:07 -04:00
Don Porter 662c253edf One more timeouts 2018-08-13 10:21:40 -04:00
Don Porter e169ee698f More CI debugging 2018-08-13 10:21:32 -04:00
Don Porter 1a63a16deb More CI debugging 2018-08-13 10:20:49 -04:00
Don Porter 1a46e2fbe2 More CI debugging 2018-08-13 10:20:18 -04:00
Don Porter 2974af0252 More timeouts 2018-08-13 10:20:05 -04:00
Don Porter 62b57a4f79 Bump up a few LTP timeouts 2018-08-13 10:20:00 -04:00
Don Porter dc40c5f809 Fix assert for SGX=1 and DEBUG=1 (Issue #97) (#98)
* Try adding CI commands that will catch issues like #97.   This should cause a CI failure.  Fix to follow.

* Define __abort for the urts.

* Fix issue #97, part 2
2018-08-13 10:19:52 -04:00
Don Porter fe7a6e6934 Fix a bug introduced by commit 3a40af399d (#94)
* Fix a PAL unit test

* Make the PAL unit tests fail properly if one that should work doesn't.

* Get consistent indexes with and without graphene ipc module

* A little debugging output for ltp flakiness.  Seems to make things a little more stable.

* A bugfix for issue #80. The migration is not complete when a file-backed VMA is larger than
the file size, but the file size is not page-aligned. In Linux, if the file size is smaller
than the mapped memory, accessing the remaining memory that is not backed by the file will
trigger a SIGBUS. However, it is fine to access the remaining memory within the last page that
still overlaps with file, and won't trigger a SIGBUS. This area is commonly used in ELF binary
to store uninitialized, static variables. To improve fork latency, Graphene chooses to
truncate the migration size as the file size, but missed the memory which belongs to the last
file-backed page. The migration size should be aligned up to the page size.

* Fix assertion lines in PAL and LibOS

* Make sure the return code is correct for the LTP script, bump up some timeouts

* Only run gipc tests when the module is loaded, for the purposes of getting CI to work.

* Build fix
2018-08-13 10:19:44 -04:00
Neil Davis e623e4e710 Jenkins (#91)
* Added Jenkinsfile for testing

* Got rid of sudo

* Adding new Jenkinsfile for testing Jenkins

* Edit to Test stage for Python pipeline

* Fixed a name typo

* Added new line to check the directory in the new stage

* Add commands to return to directory to execute files

* Added sudo to execute manifest file

* Testing a modifier

* Fixed option from sudo to sh

* Got rid of sh options flag

* Rewrote commands to run the Python code

* Testing to see if ./python-manifest executes

* Testing to see if ./python-manifest executes

* Testing Jenkinsfile-Python

* Testing Jenkinsfile-Python

* Testing Jenkins-Python

* Testing Jenkins-Python

* Checking how sh commands work

* Initializing make

* Testing file

* Testing file

* Testing file

* Jenkinsfile for LMBench tests

* Jenkinsfile for GCC

* Jenkinsfile for GCC

* Jenkinsfile for GCC

* Jenkinsfile for GCC

* Got rid of ./gcc-huge.manifest

* First SGX file for Python

* Adding lines to support keys

* Adding lines to support keys

* Chanaged the ordering to make

* Chanaged the ordering to make

* Checking variables in Makefile

* Add a print statement

* Print out SAVED and BUILD Target variables

* Print out SAVED in a different part of the code in order to see output in a different context

* Change print location of SAVED_TARGET

* Change print location of SAVED_TARGET

* Change print location of SAVED_TARGET

* Edit Makefile to see if SAVED_TARGET is assigned when not running SGX=1 *this is only meant to run the build stage*

* Testing Jenkinsfile after installing Intel SGX

* Got rid of ifndef for SAVED_TARGET

* Adding an environmental variable

* Adding an environmental variable

* Removed environment variable

* Testing use of 'sh' commands

* Testing use of 'sh' commands

* Testing use of 'sh' commands

* Changed ordering of sh executions

* Add regression to make for Python files

* Add regression to make for all files

* Removed the 'regression' terms after make, running 'make regression' didn't work

* Initializing Apache Jenkinsfile

* Initializing Lighttpd Jenkinsfile

* Initializing Lighttpd Jenkinsfile

* Added regression

* Run regression

* Moved all Jenkinsfiles into their own directory

* Added LMBench into the code for testing

* Edited error in specifying directory for LMBench

* Added background process for the server

* Added 'make regression'

* Changed client execution file on line 18

* Added background process for Apache

* Output the host and port after bulding Lighttpd

* Removed specifying the IP and Port

* Added output of a manifest file

* Added output of a manifest file

* Added output of lighttpd-server.conf file

* Combined IP address and port on line 18

* Added tests for lighttpd and apache

* Removed unnecessary files and added files to ignore in .gitignore

* Modifications to Makefile

* Removed logging in Pal/src/Makefile

* Removed my Makefile

* Put Makefile back in

* Reinserted contents of Makefile from main repository

* Revert "Reinserted contents of Makefile from main repository"

This reverts commit d36508709ab38489ae1fc939859e7952b8d56cd9.

* Added new tests for non-SGX Jenkinsfile

* Added DEBUG=1 to make command

* Fixed Makefile spacing errors
2018-08-13 10:18:18 -04:00
Chia-Che Tsai c187d67e31 add a python test script 2018-08-13 10:18:05 -04:00
Chia-Che Tsai 929e89fe95 update regression scripts 2018-08-13 10:17:59 -04:00
Chia-Che Tsai bb61416ff7 update regression scripts 2018-08-13 10:17:55 -04:00
Chia-Che Tsai a7d8c070dc completely deprecate sgx.heap_min 2018-08-13 10:17:49 -04:00
Chia-Che Tsai ff7a4b9a4f update openjdk and python scripts 2018-08-13 10:17:44 -04:00
Chia-Che Tsai 9a300fa06c update openjdk and python scripts 2018-08-13 10:17:39 -04:00
Chia-Che Tsai e271d2c95e reverse changes to the OpenJDK 7 directory 2018-08-13 10:17:26 -04:00
Don Porter 77d5e57563 Fix a few library versions for the latest 16.04. We probably want a better way to handle this across systems. 2018-08-13 10:16:30 -04:00
Chia-Che Tsai 346a62f163 Make the default version of linux sgx driver 1.9. Previously we confuse as 1.8. 2018-08-13 10:16:19 -04:00
Don Porter 81c78dde43 Drop heap_min. 2018-08-13 10:15:37 -04:00