Commit Graph
57 Commits
Author SHA1 Message Date
Stefan Berger 17b1245270 [Examples] Python: adapt Python constants for Fedora
Adapt the python constants so that python-simple also works on
Fedora 31. python-scipy-insecure misses some shared libraries on
Fedora 31, so it does not work there yet.
2020-05-01 20:54:04 +00:00
Stefan Berger 741f5f7cd4 [Examples] Python: move Python constants to Scripts/Makefile.python 2020-05-01 20:54:04 +00:00
Stefan Berger f7f89c2ed2 [Makefiles] Make Graphene compileable and testable on Fedora
Adapt Scripts/Makefile.configs so that we can build and test on
Fedora. Most of the tests in Examples are now also runable on
Fedora. Also add a dependency installation target for Fedora to
TensorFlow example.
2020-05-01 20:54:04 +00:00
Stefan Berger 206eb81eec [Makefiles] Get arch and distro specific vars from Makefile.configs
Extend Makefile.configs and define several variables for make to use
derived from 'gcc -dumpmachine'. In particular:
- ARCH as the architecture, e.g., x86_64
- ARCH_LONG as the long version of the architecture, e.g., x86_64-linux-gnu
- ARCH_LIBDIR as the directory where libraries are located,
  e.g., /lib/x86_64-linux-gnu

In Makefiles and manifest templates, replace the hard-coded
x86_64-linux_gnu and /lib/x86_64-linux-gnu through these variables.
Extend the already existing sed scripts to replace the necessary
variables.
2020-05-01 20:54:04 +00:00
Michał Kowalczyk 4f57ada563 [Pal] Clean up argv handling and PAL invocation
This is a preparation for even more clean-ups and bugfixes, which are
required by protected argv+envp implementation.
2020-04-20 22:00:58 +02:00
Michał Kowalczyk a783fa2f4b Remove unused profiling system 2020-04-13 18:59:03 +02:00
Isaku Yamahata e0dc66bcf6 [Make] Fix Makefile.Host message when run with SGX_RUN=1
Remove leading tab to fix the following error:

> Scripts//Makefile.Host:16: *** recipe commences before first target. Stop.
2020-02-29 02:54:48 +01:00
borysp f2b1d194a1 [Make] Remove std=gnu99 from tests 2020-02-14 05:16:50 +01:00
Isaku Yamahata df358b8ba8 [Make] Consistently use $(RM) instead of rm -f 2020-02-12 01:09:21 +01:00
Isaku Yamahata 0a7e3dd127 [LibOS,Pal] Move header files from Pal/lib/ under Pal/include/ 2020-02-05 23:21:06 -08:00
Isaku Yamahata 07722756e5 [Scripts] Include dependency file for .c in Makefile.Test 2020-02-05 23:21:06 -08:00
Isaku Yamahata cff1146892 [LibOS,Pal] Modify Makefiles' logic to simplify checks on SYS
The goal of `$(SYS)` check in Makefiles is to skip all targets on
unsupported systems. This commit defines a default goal `all` as
a no-op for simplicity.
2020-02-05 23:21:06 -08:00
Isaku Yamahata fc0b5c4012 [LibOS,Pal] remove unnecessary .PHONY: default in Makefiles 2020-02-05 23:21:05 -08:00
Isaku Yamahata 647a8f3953 [LibOS,Pal] Consolidate common CFLAGS in Makefile.configs 2020-02-05 23:21:05 -08:00
Isaku Yamahata 72ddedc065 [Scripts] Remove PAL_LOADER variable from Makefile.test 2020-02-05 23:21:05 -08:00
Isaku Yamahata 29d907a573 [LibOS,Pal] Consolidate DEBUG, WERROR, PROFILE in Makefile.configs 2020-02-05 23:21:05 -08:00
Isaku Yamahata 88ea5164d6 [LibOS,Pal] Makefiles: use += instead of = in CFLAGS/LDFLAGS/etc 2020-02-05 23:21:05 -08:00
Isaku Yamahata 86afe662f3 [Pal/Linux-SGX] Add rule cmulti to build executable from multiple .o files 2020-02-05 23:21:05 -08:00
Isaku Yamahata 50bdec8a23 [LibOS,Pal] Move Makefile.Host under Scripts/ 2020-02-05 23:21:05 -08:00
Isaku Yamahata 419b5c5a11 [Pal] Introduce common Makefile.manifest and remove Pal/src/Makefile.Test 2020-02-05 23:21:05 -08:00
Isaku Yamahata 08ae3a241c [Pal] Use expand_target_to_{sig,sgx,token} instead of expand_target() 2020-02-05 23:21:05 -08:00
Isaku Yamahata 35dd8c9423 [LibOS,Pal] Consolidate export DEBUG in Makefile.configs 2020-02-05 23:21:05 -08:00
Isaku Yamahata 0912f20ccf [LibOS,Pal] Unify the setting of CC and remove raw gcc invocations 2020-02-05 23:21:05 -08:00
Isaku Yamahata dd086816b5 [LibOS,Pal] Define AR and ARFLAGS properly 2020-02-05 23:21:05 -08:00
Isaku Yamahata 9ef2251d66 [LibOS,Pal,Scripts] Add and use recipe to create manifest from template 2020-02-05 23:21:05 -08:00
Isaku Yamahata 732712e23e [LibOS,Pal] Move makefile libraries under Scripts/
This commit moves Makefile.configs, Makefile.rules, and Makefile.Test
under Scripts/ and adjusts their includes.
2020-02-05 23:21:04 -08:00
Simon Gaiser 0daeed847f [Jenkins] Check for missing gitignores and fix found issues 2020-02-05 02:46:13 +01:00
Simon Gaiser 9052837dfc [Make] Improve download handling
Instead of implementing downloading of external resources in every
Makefile again, use one script. This script adds the following
features:

 - Always check the download against a known SHA-256 hash.

 - Support caching of downloaded resources (set DL_CACHE=/some/dir).

 - Allow offline builds if all files are cached. If DL_OFFLINE=true the
   build will never attempt to download anything.
2020-02-05 00:41:10 +01:00
Simon Gaiser 75619fe760 [Scripts/clean-check] Ignore git repos
Newer git versions update some cache in .git/index even on read-only
calls (like git status). So ignore any changes in .git when checking
make clean.
2020-02-04 01:15:59 -08:00
Simon Gaiser 7377a787bc [Makefile] Add distclean target
Before, there was no target that cleaned up downloaded sources. This
commit adds a distclean target to match the 'apps' Makefiles.
2020-02-04 01:15:08 +01:00
Simon Gaiser 2d29f7aaeb [Makefile] Drop SGX_RUN
For detection of SGX/non-SGX (for example in regression tests) always
use the SGX environment variable. To generate launch/EINIT tokens use
the new 'sgx-tokens' Make target.
2019-09-27 12:38:14 +02:00
Simon Gaiser 8dd19d28fb [Makefile] Check make clean on Jenkins 2019-09-27 03:45:21 +02:00
Simon Gaiser 13cac1df92 [Makefile] Fix missing cleans 2019-09-27 03:45:21 +02:00
Michał Kowalczyk 5ec5e2f24c Delete all reference monitor residues 2019-07-31 02:30:47 +02:00
Wojtek Porczyk 01f3553a25 [LibOS, Pal] Reorganize unit tests
- Deduplicate much of setup code.
- Allow running non-sandbox and sandbox code in single run.
- Use pytest.
- Generate JUnit-XML file for Jenkins.
- Document running a subset of regression tests.
2019-07-29 19:38:14 +02:00
Isaku Yamahata a84fe47843 Clean up .gitignore files
Move generic patterns to the top-level .gitignore and add some more
patterns for convenience.
2019-05-14 03:03:49 +02:00
Simon Gaiser c482288936 [Scripts/regression] Improve subprocess handling
This fixes two problems with subprocess handling:
 1. Use subprocess.communicate to avoid that the pipe can fill up.
 2. Use a process group to ensure that we also kill the subprocess'
    children in case of a timeout.

Note: This does not cover the case that the child returns successfully
but leaves other child processes running. AFAICS this is not easy since
subprocess.communicate() wait(2)s for the process so the pid is no
longer guaranteed to be valid to kill. So we leave this to the outside
environment (Jenkins or whatever).
2019-04-26 17:39:20 +02:00
Simon Gaiser 0a0babc26a [*/regression] Remove trailing whitespace in Python files 2019-04-15 17:27:30 +02:00
Simon Gaiser 79d6fb27ec [*/regression] Migrate to Python 3 2019-04-15 17:27:30 +02:00
Simon Gaiser 3d60004a8c [*/regression] Drop unused shebang
The regression python scripts are not marked executable. Also they
expect that the Makefile has setup the environment correctly so just
marking them executable would not be useful. So drop the unused shebang
to avoid encoding the interpreter in multiple places.

Script/regression.py is a library so it also doesn't need a shebang.
2019-04-15 17:24:27 +02:00
Simon Gaiser 483cde8ab7 [Scripts/regression] Treat timeouts as failure 2019-04-12 19:49:17 +02:00
Simon Gaiser f8d516bdb4 [Scripts/regression] Log ignored failure in one line
This makes parsing the output easier visually as well as by machine.
2019-03-18 17:03:15 +01:00
Simon Gaiser 9d1b70bd57 [Scripts/regression] Rename 'flaky' to 'ignore_failure'
There are different reasons why a test failure should be ignored, so
rename the option to avoid confusion if the test isn't flaky.
2019-03-18 17:03:15 +01:00
Michał Kowalczyk c9eb1d84e1 Fix Bash shebangs compatibility 2019-02-05 17:59:25 +01:00
Michał Kowalczyk 91f1301bd3 Fix Python shebangs compatibility 2019-02-05 17:59:25 +01:00
Chia-Che Tsai 6e3c5aea56 deprecating .packed directories (#149)
.packed directories are originally created for testing on non-Linux platforms. The usage is mostly obsolete right now, and the binaries are completely out-dated. A better strategy would be to create a hook in the github to release binary packages.
2017-12-21 11:13:53 -05:00
Chia-Che Tsai 84f81d8f13 Lots of bugfixes (#145)
- Fix a severe bug caused by merging #87. The patch does not check the return values from get_config_entries_size(), while the return values can easily be negative (-PAL_ERROR_INVAL). Any usage of this function should carefully check the return values and also pass the size into get_config_entries() to prevent buffer overflow.
- Fix a minor bug in handling IPC disconnection. The callback ipc_child_exit() never receives a "term_signal" parameter. The callback is called when thepipe to a child process unexpectedly terminates, which can only mean the child process has crashed. I believe the proper signal to send is SIGKILL.
- Remove a compilation warning caused by type mismatch of __malloc().
- Allow binding TCP socket to ANY address
- enable LD_PRELOAD in glibc
- disable output buffering in LTP tests to reveal omitted passes
- Remove a double-unlock in shim_async.c
- Replace all int with size_t or ssize_t in Graphene configuration API
- Add new passed LTP tests
2017-12-18 16:03:36 -08:00
Chia-Che Tsai 3aaf591fc5 Fixing a bunch of memory leaking issues (#86)
* add memusg to pal_loader script

* Stop bleeding PAL handles.

Deprecating DkOjectReference and reference counting in PAL handle.
Deprecating DkSemaphoreDestroy and DkEventDestroy (replaced by DkObjectClose).
Cleaning unused PAL handles in the library OS.
Adding a heap tracing feature to profile usage of PAL handles.

* fix a bug in SGX mode that mapping untrusted files into memory never got free by DkVirtualMemoryFree()

* adding lighttpd SSL option

* fixing the freeing convention of PAL handles; On SGX, event and mutex handles need to be freed seperately.

* changing how mutexes and events are allocated on SGX

* fixing GCC regression tests (for both Linux and SGX)

* fix a double-free problem of the first thread handle
2017-12-08 10:46:55 -08:00
Don Porter f67fe09124 Fix a bug introduced by commit 3a40af399d (#94)
* Fix a PAL unit test

* Make the PAL unit tests fail properly if one that should work doesn't.

* Get consistent indexes with and without graphene ipc module

* A little debugging output for ltp flakiness.  Seems to make things a little more stable.

* A bugfix for issue #80. The migration is not complete when a file-backed VMA is larger than
the file size, but the file size is not page-aligned. In Linux, if the file size is smaller
than the mapped memory, accessing the remaining memory that is not backed by the file will
trigger a SIGBUS. However, it is fine to access the remaining memory within the last page that
still overlaps with file, and won't trigger a SIGBUS. This area is commonly used in ELF binary
to store uninitialized, static variables. To improve fork latency, Graphene chooses to
truncate the migration size as the file size, but missed the memory which belongs to the last
file-backed page. The migration size should be aligned up to the page size.

* Fix assertion lines in PAL and LibOS

* Make sure the return code is correct for the LTP script, bump up some timeouts

* Only run gipc tests when the module is loaded, for the purposes of getting CI to work.

* Build fix
2017-11-10 11:46:11 +00:00
Don Porter b3da2aff67 Add a unit test and clearer debug print for issue #39 2017-01-21 13:59:37 -05:00