From d53729b20142275aa8728cef4aaca3ffbee4f88b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Kowalczyk?= Date: Thu, 3 Dec 2020 01:53:01 +0100 Subject: [PATCH] [Pal] Rework manifest loading This is a major refactor of the way manifests are loaded and handled, which will be followed by a complete rework of the loader code (which will include e.g. centralized config). Changes/fixes: - Huge part of manifest handling was refactored and untangled. - Starting without a manifest is now disallowed. This was actually accidentally broken for some time and no one complained. It also makes little sense in practice and in Graphene's overall design, e.g. it conflicts with protected argv. - Now we only allow starting by giving the executable, not manifest (the magic resolution logic was removed). - Now manifests are sent over pipes between parent and children, instead of children finding and loading them on their own. This is a preparation for the upcoming centralized manifests change. - Previously manifests were parsed 2 times on Linux and 3 times on Linux-SGX (by untrusted PAL, trusted PAL and LibOS). This is now fixed. - The common `pal_main()` now requires that the backend-specific PAL loader loads the manifest before calling it. SGX code already has to do it (for proper initialization), so let's unify this interface for all PALs. - Fix for a PAL crash when manifest size was divisible by page size (sic!). NULL termination was missing, but most of the time the padding to page size saved Graphene from crashing. --- Documentation/devel/debugging.rst | 4 +- Documentation/devel/performance.rst | 2 +- Documentation/manpages/pal_loader.rst | 2 +- Documentation/pal/host-abi.rst | 22 +- Documentation/tutorials/pytorch/index.rst | 14 +- Examples/apache/Makefile | 4 +- Examples/bash/Makefile | 6 +- Examples/bash/README.md | 8 +- Examples/busybox/README.md | 16 - Examples/busybox/busybox.manifest.template | 4 - Examples/curl/Makefile | 2 +- Examples/gcc/Makefile | 6 +- Examples/lighttpd/Makefile | 2 +- Examples/lighttpd/lighttpd.manifest.template | 2 +- Examples/nginx/Makefile | 2 +- Examples/nginx/README.md | 4 +- Examples/nodejs-express-server/Makefile | 2 +- Examples/nodejs/Makefile | 2 +- Examples/python-scipy-insecure/Makefile | 4 +- Examples/python-scipy-insecure/README.md | 8 +- Examples/python-simple/README.md | 8 +- Examples/python-simple/run-tests.sh | 8 +- Examples/pytorch/README.md | 4 +- Examples/r/Makefile | 2 +- Examples/r/README.md | 8 +- Examples/ra-tls-mbedtls/README.md | 4 +- LibOS/shim/include/shim_internal.h | 1 - LibOS/shim/src/shim_init.c | 62 +-- Pal/include/host/Linux-common/linux_utils.h | 2 + Pal/include/pal/pal.h | 13 +- Pal/regression/.gitignore | 2 - Pal/regression/Bootstrap.c | 5 - Pal/regression/Bootstrap2.c | 24 - Pal/regression/Bootstrap2.manifest.template | 2 - Pal/regression/Bootstrap4.manifest.template | 1 - Pal/regression/Makefile | 6 +- Pal/regression/Process.c | 5 - Pal/regression/test_pal.py | 14 - Pal/src/db_main.c | 243 +++------ Pal/src/host/Linux-SGX/Makefile | 2 +- Pal/src/host/Linux-SGX/db_main.c | 20 +- Pal/src/host/Linux-SGX/db_misc.c | 2 - Pal/src/host/Linux-SGX/enclave_framework.c | 1 - Pal/src/host/Linux-SGX/enclave_pf.c | 3 - Pal/src/host/Linux-SGX/pal_linux.h | 3 +- Pal/src/host/Linux-SGX/pal_security.h | 2 - Pal/src/host/Linux-SGX/sgx_enclave.c | 33 +- Pal/src/host/Linux-SGX/sgx_framework.c | 3 + Pal/src/host/Linux-SGX/sgx_internal.h | 11 +- Pal/src/host/Linux-SGX/sgx_main.c | 531 ++++++++----------- Pal/src/host/Linux-SGX/sgx_process.c | 63 ++- Pal/src/host/Linux-common/file_utils.c | 69 +++ Pal/src/host/Linux/Makefile | 1 + Pal/src/host/Linux/db_main.c | 58 +- Pal/src/host/Linux/db_process.c | 133 +++-- Pal/src/host/Linux/pal_linux.h | 2 +- Pal/src/pal_internal.h | 15 +- Pal/src/slab.c | 3 +- README.rst | 17 +- Scripts/regression.py | 3 - Tools/gsc/templates/apploader.template | 4 +- python/graphenelibos/sgx_sign.py | 30 +- 62 files changed, 652 insertions(+), 887 deletions(-) delete mode 100644 Pal/regression/Bootstrap2.c delete mode 100644 Pal/regression/Bootstrap2.manifest.template delete mode 100644 Pal/regression/Bootstrap4.manifest.template create mode 100644 Pal/src/host/Linux-common/file_utils.c diff --git a/Documentation/devel/debugging.rst b/Documentation/devel/debugging.rst index 45985121..1b5847b2 100644 --- a/Documentation/devel/debugging.rst +++ b/Documentation/devel/debugging.rst @@ -19,7 +19,7 @@ To build Graphene with debug symbols, the source code needs to be compiled with To run Graphene with GDB, use the following command to run your application:: - GDB=1 [Graphene Directory]/Runtime/pal_loader [executable|manifest] [arguments] + GDB=1 [Graphene Directory]/Runtime/pal_loader [executable] [arguments] Debugging with SGX support -------------------------- @@ -46,4 +46,4 @@ run the following commands:: To run Graphene with GDB, use the Graphene loader (``pal_loader``) and specify ``GDB=1``:: - GDB=1 SGX=1 [Graphene Directory]/Runtime/pal_loader [executable|manifest] [arguments] + GDB=1 SGX=1 [Graphene Directory]/Runtime/pal_loader [executable] [arguments] diff --git a/Documentation/devel/performance.rst b/Documentation/devel/performance.rst index 4fd05829..3c27aaba 100644 --- a/Documentation/devel/performance.rst +++ b/Documentation/devel/performance.rst @@ -472,7 +472,7 @@ Recording samples with ``perf record`` To record (saves ``perf.data``):: - perf record ./pal_loader manifest + perf record ./pal_loader executable To view the report for ``perf.data``:: diff --git a/Documentation/manpages/pal_loader.rst b/Documentation/manpages/pal_loader.rst index 2f6a8012..fc553ba0 100644 --- a/Documentation/manpages/pal_loader.rst +++ b/Documentation/manpages/pal_loader.rst @@ -11,7 +11,7 @@ Synopsis ======== -:command:`pal_loader` [SGX] [GDB] {<*MANIFEST*> | <*EXECUTABLE*>} [<*ARGS*> ...] +:command:`pal_loader` [SGX] [GDB] {<*EXECUTABLE*>} [<*ARGS*> ...] Description =========== diff --git a/Documentation/pal/host-abi.rst b/Documentation/pal/host-abi.rst index 0c22a2c3..36a15836 100644 --- a/Documentation/pal/host-abi.rst +++ b/Documentation/pal/host-abi.rst @@ -28,26 +28,18 @@ is described in :doc:`../manifest-syntax`. Manifest and executable loading ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The PAL loader supports multiple ways of locating the manifest and executable. To run a program in Graphene properly, the PAL loader generally requires both a -manifest and an executable, although it is possible to load with only one of -them. The user shall specify either the manifest or the executable to load in -the command line, and the PAL loader will try to locate the other based on the -file name. +manifest and an executable. The user shall specify the executable to load in +the command line, and the PAL loader will try to locate the manifest based on +the file name. Precisely, the loading rules for the manifest and executable are as follows: -#. The first argument given to :program:`pal_loader` can be either a manifest - file or an executable. -#. If an executable is given to the command line, the loader will search for the +#. The first argument given to :program:`pal_loader` has to be an executable. +#. The loader will search for the manifest in the following order: the same file name as the executable with - a `.manifest` or `.manifest.sgx` extension, a `manifest` file without any - extension, or no manifest at all. -#. If a manifest is given to the command line, then the manifest will be used - to infer the executable. The potential executable file has the same file name - as the manifest file except it doesn't have the `.manifest` or - `.manifest.sgx` extension. - + a ``.manifest`` or ``.manifest.sgx`` extension, or ``manifest`` file + without any extension. Data types and variables ------------------------ diff --git a/Documentation/tutorials/pytorch/index.rst b/Documentation/tutorials/pytorch/index.rst index 1c8c2f98..a6682e64 100644 --- a/Documentation/tutorials/pytorch/index.rst +++ b/Documentation/tutorials/pytorch/index.rst @@ -186,8 +186,8 @@ Python as the actual executable). For illustrative purposes, we will look at only a few entries of the file. Note that we can simply ignore SGX-specific keys (starting with the ``sgx.`` prefix) for our non-SGX run. -Notice that the manifest file is not fully secure because it propagates -untrusted command-line arguments and environment variables into the enclave. We +Notice that the manifest file is not secure because it propagates untrusted +command-line arguments and environment variables into the enclave. We keep these work-arounds in this tutorial for simplicity, but this configuration must not be used in production:: @@ -229,11 +229,11 @@ This command will autogenerate a couple new files: also create a symlink to the Python binary, with the name corresponding to the manifest. -Now, launch Graphene with the generated manifest via ``pal_loader``. You can -simply append the arguments after the manifest name. Our example takes +Now, launch Graphene via ``pal_loader``. You can simply append the arguments +after the executable name. Our example takes ``pytorchexample.py`` as an argument:: - ./pal_loader python3.manifest pytorchexample.py + ./pal_loader ./python3 pytorchexample.py That's it. You have run the PyTorch example with Graphene. You can check ``result.txt`` to make sure it ran correctly. @@ -306,7 +306,7 @@ After running this command and building all the required files, we can simply set ``SGX=1`` environment variable and use ``pal_loader`` to launch the PyTorch workload inside an SGX enclave:: - SGX=1 ./pal_loader python3.manifest.sgx pytorchexample.py + SGX=1 ./pal_loader ./python3 pytorchexample.py It will run exactly the same Python script but inside the SGX enclave. Again, you can verify that PyTorch ran correctly by examining ``result.txt``. @@ -538,7 +538,7 @@ We are ready to run the end-to-end PyTorch example. Notice that we didn't change a line of code in the Python script. Moreover, we can run it with exactly the same command used in the previous section:: - SGX=1 ./pal_loader python3.manifest pytorchexample.py + SGX=1 ./pal_loader ./python3 pytorchexample.py This should run PyTorch with encrypted input files and generate the encrypted ``result.txt`` output file. Note that we already launched the secret diff --git a/Examples/apache/Makefile b/Examples/apache/Makefile index d7ece0c6..e1797d81 100644 --- a/Examples/apache/Makefile +++ b/Examples/apache/Makefile @@ -256,7 +256,7 @@ start-native-server: all clean-server .PHONY: start-graphene-server start-graphene-server: all clean-server @echo "Listen on $(LISTEN_HOST):$(LISTEN_PORT)" - $(PREFIX) ./pal_loader httpd.manifest -D FOREGROUND \ + $(PREFIX) ./pal_loader ./httpd -D FOREGROUND \ -f conf/httpd-graphene.conf \ -C "LoadModule mpm_prefork_module modules/mod_mpm_prefork.so" \ -C "Listen $(LISTEN_HOST):$(LISTEN_PORT)" \ @@ -276,7 +276,7 @@ start-native-multithreaded-server: all clean-server .PHONY: start-graphene-multithreaded-server start-graphene-multithreaded-server: all clean-server @echo "Listen on $(LISTEN_HOST):$(LISTEN_PORT)" - $(PREFIX) ./pal_loader httpd.manifest -D FOREGROUND \ + $(PREFIX) ./pal_loader ./httpd -D FOREGROUND \ -f conf/httpd-graphene.conf \ -C "LoadModule mpm_worker_module modules/mod_mpm_worker.so" \ -C "Listen $(LISTEN_HOST):$(LISTEN_PORT)" \ diff --git a/Examples/bash/Makefile b/Examples/bash/Makefile index 543205d0..8dcd4514 100644 --- a/Examples/bash/Makefile +++ b/Examples/bash/Makefile @@ -123,18 +123,18 @@ pal_loader: regression: all @mkdir -p scripts/testdir - ./pal_loader bash.manifest -c "ls" > OUTPUT + ./pal_loader ./bash -c "ls" > OUTPUT @grep -q "Makefile" OUTPUT && echo "[ Success 1/6 ]" @rm OUTPUT - ./pal_loader bash.manifest -c "cd scripts && bash bash_test.sh 1" > OUTPUT + ./pal_loader ./bash -c "cd scripts && bash bash_test.sh 1" > OUTPUT @grep -q "hello 1" OUTPUT && echo "[ Success 2/6 ]" @grep -q "createdfile" OUTPUT && echo "[ Success 3/6 ]" @grep -q "somefile" OUTPUT && echo "[ Success 4/6 ]" @grep -q "current date" OUTPUT && echo "[ Success 5/6 ]" @rm OUTPUT - ./pal_loader bash.manifest -c "cd scripts && bash bash_test.sh 3" > OUTPUT + ./pal_loader ./bash -c "cd scripts && bash bash_test.sh 3" > OUTPUT @grep -q "hello 3" OUTPUT && echo "[ Success 6/6 ]" @rm OUTPUT diff --git a/Examples/bash/README.md b/Examples/bash/README.md index 7d68ee83..4b1661fa 100644 --- a/Examples/bash/README.md +++ b/Examples/bash/README.md @@ -20,12 +20,12 @@ Here's an example of running Bash scripts under Graphene: Without SGX: ``` -./pal_loader bash.manifest -c "ls" -./pal_loader bash.manifest -c "cd scripts && bash bash_test.sh 2" +./pal_loader ./bash -c "ls" +./pal_loader ./bash -c "cd scripts && bash bash_test.sh 2" ``` With SGX: ``` -SGX=1 ./pal_loader bash.manifest -c "ls" -SGX=1 ./pal_loader bash.manifest -c "cd scripts && bash bash_test.sh 2" +SGX=1 ./pal_loader ./bash -c "ls" +SGX=1 ./pal_loader ./bash -c "cd scripts && bash bash_test.sh 2" ``` diff --git a/Examples/busybox/README.md b/Examples/busybox/README.md index 56e85fcb..dbceeaac 100644 --- a/Examples/busybox/README.md +++ b/Examples/busybox/README.md @@ -16,27 +16,11 @@ make SGX=1 # run Busybox shell in non-SGX Graphene ./pal_loader busybox sh -# or -./pal_loader busybox.manifest sh # run Busybox shell in Graphene-SGX SGX=1 ./pal_loader busybox sh -# or -SGX=1 ./pal_loader busybox.manifest.sgx sh # now a shell session should be running e.g. typing: ls # should run program `ls` which lists current working directory ``` - -Note that busybox can also be started via a manifest file (which contains path -to the busybox binary): - -```sh - ./busybox.manifest sh # to run a shell - ./busybox.manifest ls -l # to list local directory - - # or under SGX: - ./busybox.manifest.sgx sh # to run a shell - ./busybox.manifest.sgx ls -l # to list local directory -``` diff --git a/Examples/busybox/busybox.manifest.template b/Examples/busybox/busybox.manifest.template index 9447af72..257bb8fb 100644 --- a/Examples/busybox/busybox.manifest.template +++ b/Examples/busybox/busybox.manifest.template @@ -4,10 +4,6 @@ ################################## GRAPHENE ################################### -# Name of the executable e.g. as visible through argv[0] when ran as manifest -# file (`./pal_loader busybox.manifest`). -loader.argv0_override = "busybox" - # Read application arguments directly from the command line. Don't use this on production! loader.insecure__use_cmdline_argv = 1 diff --git a/Examples/curl/Makefile b/Examples/curl/Makefile index f104a5f3..53bdeb1d 100644 --- a/Examples/curl/Makefile +++ b/Examples/curl/Makefile @@ -84,7 +84,7 @@ pal_loader: check: all (cd test-docroot; exec python3 -m http.server -b 127.0.0.1 19111) & httpd_pid=$$!; \ sleep 1; \ - ./pal_loader curl.manifest http://127.0.0.1:19111/ > OUTPUT; rc=$$?; \ + ./pal_loader ./curl http://127.0.0.1:19111/ > OUTPUT; rc=$$?; \ kill $$httpd_pid; exit $$rc @grep -q "Hello World" OUTPUT && echo "[ Success 1/1 ]" diff --git a/Examples/gcc/Makefile b/Examples/gcc/Makefile index 60482a8d..5fb69237 100644 --- a/Examples/gcc/Makefile +++ b/Examples/gcc/Makefile @@ -115,13 +115,13 @@ cc1-ldd: cc1 .PHONY: check check: all @echo "\n\nCompiling hello.c..." - ./pal_loader gcc.manifest test_files/helloworld.c -o test_files/hello + ./pal_loader ./gcc test_files/helloworld.c -o test_files/hello @chmod 755 test_files/hello -./test_files/hello $(RM) test_files/hello @echo "\n\nCompiling bzip2.c..." - ./pal_loader ./gcc.manifest test_files/bzip2.c -o test_files/bzip2 + ./pal_loader ./gcc test_files/bzip2.c -o test_files/bzip2 @chmod 755 test_files/bzip2 $(RM) bzip2.tmp @cp -f test_files/bzip2 test_files/bzip2.copy @@ -131,7 +131,7 @@ check: all $(RM) test_files/bzip2 test_file/bzip2.copy @echo "\n\nCompiling gzip.c..." - ./pal_loader ./gcc.manifest test_files/gzip.c -o test_files/gzip + ./pal_loader ./gcc test_files/gzip.c -o test_files/gzip @chmod 755 test_files/gzip @cp -f test_files/gzip test_files/gzip.copy ./test_files/gzip test_files/gzip.copy diff --git a/Examples/lighttpd/Makefile b/Examples/lighttpd/Makefile index 51ab6a95..d9f25560 100644 --- a/Examples/lighttpd/Makefile +++ b/Examples/lighttpd/Makefile @@ -164,7 +164,7 @@ start-native-server: all .PHONY: start-graphene-server start-graphene-server: all - ./pal_loader lighttpd.manifest -D -m $(INSTALL_DIR)/lib -f lighttpd.conf + ./pal_loader ./lighttpd -D -m $(INSTALL_DIR)/lib -f lighttpd.conf .PHONY: clean clean: diff --git a/Examples/lighttpd/lighttpd.manifest.template b/Examples/lighttpd/lighttpd.manifest.template index 6d09eecd..29428b5c 100644 --- a/Examples/lighttpd/lighttpd.manifest.template +++ b/Examples/lighttpd/lighttpd.manifest.template @@ -4,7 +4,7 @@ # # lighttpd must be run with the pal_loader: # -# ./pal_loader lighttpd.manifest