From 6c7ff2d3a68645713292a32d65858284acfec806 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Micha=C5=82=20Kowalczyk?= Date: Tue, 24 Mar 2020 14:59:37 +0100 Subject: [PATCH] Migrate and remove test/apps submodule We decided to merge the sample app integrations submodule back because working with git submodules turned out to be really painful. The only blocker for this was the fact, that previously it contained a lot of binary blobs and copy-pasted sources, but this was cleaned up recently. Credits: (authors of particular integration examples, extracted from commits and PR history in https://github.com/oscarlab/graphene-tests) apache: Chia-Che Tsai , Dmitrii Kuvaiskii bash: Chia-Che Tsai , Dmitrii Kuvaiskii blender: borysp busybox: borysp capnproto: Dmitrii Kuvaiskii curl: Dmitrii Kuvaiskii gcc: Thomas Knauth lighttpd: Chia-Che Tsai , Thomas Knauth lmbench: Chia-Che Tsai memcached: Dmitrii Kuvaiskii nginx: Dmitrii Kuvaiskii nodejs: jack.wxz nodejs-express-server: Eduardo Rodriguez openvino: Dmitrii Kuvaiskii python-scipy-insecure: Chia-Che Tsai , Dmitrii Kuvaiskii python-simple: Chia-Che Tsai , Dmitrii Kuvaiskii pytorch: Thomas Knauth r: Chia-Che Tsai redis: Dmitrii Kuvaiskii tensorflow: Thomas Knauth LTP was moved to LibOS/shim/test/ltp. It was recently rewritten by Wojtek Porczyk . --- Examples/.gitignore | 9 + Examples/README.md | 66 + Examples/apache/.gitignore | 7 + Examples/apache/Makefile | 279 + Examples/apache/README.md | 61 + Examples/apache/benchmark-http.sh | 1 + Examples/apache/httpd.manifest.template | 114 + Examples/apache/ssl/ca_config.conf | 16 + Examples/bash/.gitignore | 2 + Examples/bash/Makefile | 140 + Examples/bash/README.md | 31 + Examples/bash/manifest.template | 80 + Examples/bash/scripts/bash_test.sh | 14 + Examples/bash/scripts/somefile | 1 + Examples/blender/.gitignore | 3 + Examples/blender/Makefile | 81 + Examples/blender/blender.manifest.template | 82 + Examples/blender/data/.gitignore | 1 + .../blender/data/scenes/simple_scene.blend | Bin 0 -> 178901 bytes Examples/blender/test_all_scenes.sh | 11 + Examples/busybox/.gitignore | 3 + Examples/busybox/Makefile | 112 + Examples/busybox/README.md | 33 + Examples/busybox/busybox.manifest.template | 139 + Examples/capnproto/.gitignore | 2 + Examples/capnproto/Makefile | 108 + Examples/capnproto/README | 26 + .../capnproto/addressbook.manifest.template | 53 + Examples/common_tools/benchmark-http.sh | 49 + Examples/curl/.gitignore | 1 + Examples/curl/Makefile | 92 + Examples/curl/README.md | 18 + Examples/curl/curl.manifest.template | 76 + Examples/curl/test-docroot/index.html | 2 + Examples/gcc/.gitignore | 4 + Examples/gcc/Makefile | 126 + Examples/gcc/README.md | 19 + Examples/gcc/as.manifest.template | 37 + Examples/gcc/cc1.manifest.template | 39 + Examples/gcc/collect2.manifest.template | 38 + Examples/gcc/gcc.manifest.template | 42 + Examples/gcc/ld-gcc-5-trusted-files | 1 + Examples/gcc/ld-gcc-7-trusted-files | 1 + Examples/gcc/ld.manifest.template | 49 + Examples/gcc/test_files/.gitignore | 1 + Examples/gcc/test_files/bzip2.c | 6998 ++++++++++++++ Examples/gcc/test_files/gzip.c | 8607 +++++++++++++++++ Examples/gcc/test_files/helloworld.c | 6 + Examples/lighttpd/.gitignore | 8 + Examples/lighttpd/Makefile | 166 + Examples/lighttpd/README.md | 49 + Examples/lighttpd/benchmark-http.sh | 1 + Examples/lighttpd/lighttpd-generic.conf | 66 + Examples/lighttpd/lighttpd.manifest.template | 86 + Examples/lmbench/.gitignore | 2 + Examples/lmbench/Makefile | 128 + Examples/lmbench/README.md | 68 + Examples/lmbench/hello.manifest.template | 41 + Examples/lmbench/lmbench-test-ci.sh | 282 + Examples/lmbench/manifest.template | 84 + Examples/lmbench/sh.manifest.template | 41 + Examples/memcached/.gitignore | 2 + Examples/memcached/Makefile | 102 + Examples/memcached/README | 33 + .../memcached/memcached.manifest.template | 198 + Examples/nginx/.gitignore | 5 + Examples/nginx/Makefile | 175 + Examples/nginx/README.md | 47 + Examples/nginx/benchmark-http.sh | 1 + Examples/nginx/nginx.manifest.template | 100 + Examples/nodejs-express-server/.gitignore | 1 + Examples/nodejs-express-server/Makefile | 91 + Examples/nodejs-express-server/README.md | 29 + Examples/nodejs-express-server/helloworld.js | 19 + .../nodejs.manifest.template | 57 + Examples/nodejs-express-server/package.json | 12 + Examples/nodejs/Makefile | 88 + Examples/nodejs/README.md | 12 + Examples/nodejs/helloworld.js | 1 + Examples/nodejs/nodejs.manifest.template | 57 + Examples/openvino/.gitignore | 3 + Examples/openvino/Makefile | 98 + Examples/openvino/README.md | 52 + Examples/openvino/images/horses.jpg | Bin 0 -> 811782 bytes Examples/openvino/openvino.manifest.template | 124 + Examples/python-scipy-insecure/.gitignore | 1 + Examples/python-scipy-insecure/Makefile | 130 + Examples/python-scipy-insecure/README.md | 57 + .../python.manifest.template | 107 + .../scripts/test-numpy.py | 20 + .../scripts/test-scipy.py | 15 + Examples/python-simple/.gitignore | 1 + Examples/python-simple/Makefile | 140 + Examples/python-simple/README.md | 58 + .../python-simple/python.manifest.template | 264 + Examples/python-simple/run-tests.sh | 28 + Examples/python-simple/scripts/benchrun.py | 116 + .../python-simple/scripts/dummy-web-server.py | 54 + Examples/python-simple/scripts/fibonacci.py | 54 + Examples/python-simple/scripts/helloworld.py | 3 + Examples/python-simple/scripts/test-http.py | 14 + Examples/pytorch/Makefile | 76 + Examples/pytorch/README.md | 40 + Examples/pytorch/classes.txt | 1000 ++ Examples/pytorch/input.jpg | Bin 0 -> 78949 bytes Examples/pytorch/pytorch.manifest.template | 146 + Examples/pytorch/pytorchexample.py | 47 + Examples/r/Makefile | 124 + Examples/r/R.manifest.template | 95 + Examples/r/README.md | 43 + Examples/r/scripts/R-benchmark-25.R | 385 + Examples/r/scripts/sample.r | 4 + Examples/r/sh.manifest.template | 41 + Examples/redis/.gitignore | 3 + Examples/redis/Makefile | 121 + Examples/redis/README | 49 + Examples/redis/redis-server.manifest.template | 156 + Examples/tensorflow/.gitignore | 2 + Examples/tensorflow/Makefile | 99 + Examples/tensorflow/README.md | 17 + .../tensorflow/label_image.manifest.template | 32 + LibOS/shim/test/apps | 1 - LibOS/shim/test/ltp/.gitignore | 2 + LibOS/shim/test/ltp/Makefile | 55 + LibOS/shim/test/ltp/Makefile.Test | 130 + LibOS/shim/test/ltp/Makefile.testcases | 24 + LibOS/shim/test/ltp/README.rst | 95 + LibOS/shim/test/ltp/contrib/conf_lint.py | 53 + LibOS/shim/test/ltp/contrib/conf_merge.py | 63 + LibOS/shim/test/ltp/contrib/conf_missing.py | 47 + .../test/ltp/contrib/conf_remove_must_pass.py | 81 + LibOS/shim/test/ltp/ltp-bug-1075.cfg | 1013 ++ LibOS/shim/test/ltp/ltp-bug-1248.cfg | 2 + LibOS/shim/test/ltp/ltp-sgx.cfg | 3781 ++++++++ LibOS/shim/test/ltp/ltp.cfg | 3276 +++++++ LibOS/shim/test/ltp/makevars.mk | 6 + LibOS/shim/test/ltp/manifest.template | 39 + LibOS/shim/test/ltp/runltp_xml.py | 611 ++ 138 files changed, 32830 insertions(+), 1 deletion(-) create mode 100644 Examples/.gitignore create mode 100644 Examples/README.md create mode 100644 Examples/apache/.gitignore create mode 100644 Examples/apache/Makefile create mode 100644 Examples/apache/README.md create mode 120000 Examples/apache/benchmark-http.sh create mode 100644 Examples/apache/httpd.manifest.template create mode 100644 Examples/apache/ssl/ca_config.conf create mode 100644 Examples/bash/.gitignore create mode 100644 Examples/bash/Makefile create mode 100644 Examples/bash/README.md create mode 100644 Examples/bash/manifest.template create mode 100755 Examples/bash/scripts/bash_test.sh create mode 100644 Examples/bash/scripts/somefile create mode 100644 Examples/blender/.gitignore create mode 100644 Examples/blender/Makefile create mode 100644 Examples/blender/blender.manifest.template create mode 100644 Examples/blender/data/.gitignore create mode 100644 Examples/blender/data/scenes/simple_scene.blend create mode 100644 Examples/blender/test_all_scenes.sh create mode 100644 Examples/busybox/.gitignore create mode 100644 Examples/busybox/Makefile create mode 100644 Examples/busybox/README.md create mode 100644 Examples/busybox/busybox.manifest.template create mode 100644 Examples/capnproto/.gitignore create mode 100644 Examples/capnproto/Makefile create mode 100644 Examples/capnproto/README create mode 100644 Examples/capnproto/addressbook.manifest.template create mode 100755 Examples/common_tools/benchmark-http.sh create mode 100644 Examples/curl/.gitignore create mode 100644 Examples/curl/Makefile create mode 100644 Examples/curl/README.md create mode 100644 Examples/curl/curl.manifest.template create mode 100644 Examples/curl/test-docroot/index.html create mode 100644 Examples/gcc/.gitignore create mode 100644 Examples/gcc/Makefile create mode 100644 Examples/gcc/README.md create mode 100644 Examples/gcc/as.manifest.template create mode 100644 Examples/gcc/cc1.manifest.template create mode 100644 Examples/gcc/collect2.manifest.template create mode 100644 Examples/gcc/gcc.manifest.template create mode 100644 Examples/gcc/ld-gcc-5-trusted-files create mode 100644 Examples/gcc/ld-gcc-7-trusted-files create mode 100644 Examples/gcc/ld.manifest.template create mode 100644 Examples/gcc/test_files/.gitignore create mode 100644 Examples/gcc/test_files/bzip2.c create mode 100644 Examples/gcc/test_files/gzip.c create mode 100644 Examples/gcc/test_files/helloworld.c create mode 100644 Examples/lighttpd/.gitignore create mode 100644 Examples/lighttpd/Makefile create mode 100644 Examples/lighttpd/README.md create mode 120000 Examples/lighttpd/benchmark-http.sh create mode 100644 Examples/lighttpd/lighttpd-generic.conf create mode 100644 Examples/lighttpd/lighttpd.manifest.template create mode 100644 Examples/lmbench/.gitignore create mode 100644 Examples/lmbench/Makefile create mode 100644 Examples/lmbench/README.md create mode 100644 Examples/lmbench/hello.manifest.template create mode 100755 Examples/lmbench/lmbench-test-ci.sh create mode 100644 Examples/lmbench/manifest.template create mode 100644 Examples/lmbench/sh.manifest.template create mode 100644 Examples/memcached/.gitignore create mode 100644 Examples/memcached/Makefile create mode 100644 Examples/memcached/README create mode 100644 Examples/memcached/memcached.manifest.template create mode 100644 Examples/nginx/.gitignore create mode 100644 Examples/nginx/Makefile create mode 100644 Examples/nginx/README.md create mode 120000 Examples/nginx/benchmark-http.sh create mode 100644 Examples/nginx/nginx.manifest.template create mode 100644 Examples/nodejs-express-server/.gitignore create mode 100644 Examples/nodejs-express-server/Makefile create mode 100644 Examples/nodejs-express-server/README.md create mode 100644 Examples/nodejs-express-server/helloworld.js create mode 100644 Examples/nodejs-express-server/nodejs.manifest.template create mode 100644 Examples/nodejs-express-server/package.json create mode 100644 Examples/nodejs/Makefile create mode 100644 Examples/nodejs/README.md create mode 100644 Examples/nodejs/helloworld.js create mode 100644 Examples/nodejs/nodejs.manifest.template create mode 100644 Examples/openvino/.gitignore create mode 100644 Examples/openvino/Makefile create mode 100644 Examples/openvino/README.md create mode 100644 Examples/openvino/images/horses.jpg create mode 100644 Examples/openvino/openvino.manifest.template create mode 100644 Examples/python-scipy-insecure/.gitignore create mode 100644 Examples/python-scipy-insecure/Makefile create mode 100644 Examples/python-scipy-insecure/README.md create mode 100644 Examples/python-scipy-insecure/python.manifest.template create mode 100644 Examples/python-scipy-insecure/scripts/test-numpy.py create mode 100644 Examples/python-scipy-insecure/scripts/test-scipy.py create mode 100644 Examples/python-simple/.gitignore create mode 100644 Examples/python-simple/Makefile create mode 100644 Examples/python-simple/README.md create mode 100644 Examples/python-simple/python.manifest.template create mode 100755 Examples/python-simple/run-tests.sh create mode 100644 Examples/python-simple/scripts/benchrun.py create mode 100644 Examples/python-simple/scripts/dummy-web-server.py create mode 100644 Examples/python-simple/scripts/fibonacci.py create mode 100644 Examples/python-simple/scripts/helloworld.py create mode 100644 Examples/python-simple/scripts/test-http.py create mode 100644 Examples/pytorch/Makefile create mode 100644 Examples/pytorch/README.md create mode 100644 Examples/pytorch/classes.txt create mode 100644 Examples/pytorch/input.jpg create mode 100644 Examples/pytorch/pytorch.manifest.template create mode 100644 Examples/pytorch/pytorchexample.py create mode 100644 Examples/r/Makefile create mode 100644 Examples/r/R.manifest.template create mode 100644 Examples/r/README.md create mode 100644 Examples/r/scripts/R-benchmark-25.R create mode 100644 Examples/r/scripts/sample.r create mode 100644 Examples/r/sh.manifest.template create mode 100644 Examples/redis/.gitignore create mode 100644 Examples/redis/Makefile create mode 100644 Examples/redis/README create mode 100644 Examples/redis/redis-server.manifest.template create mode 100644 Examples/tensorflow/.gitignore create mode 100644 Examples/tensorflow/Makefile create mode 100644 Examples/tensorflow/README.md create mode 100644 Examples/tensorflow/label_image.manifest.template delete mode 160000 LibOS/shim/test/apps create mode 100644 LibOS/shim/test/ltp/.gitignore create mode 100644 LibOS/shim/test/ltp/Makefile create mode 100644 LibOS/shim/test/ltp/Makefile.Test create mode 100644 LibOS/shim/test/ltp/Makefile.testcases create mode 100644 LibOS/shim/test/ltp/README.rst create mode 100755 LibOS/shim/test/ltp/contrib/conf_lint.py create mode 100755 LibOS/shim/test/ltp/contrib/conf_merge.py create mode 100755 LibOS/shim/test/ltp/contrib/conf_missing.py create mode 100755 LibOS/shim/test/ltp/contrib/conf_remove_must_pass.py create mode 100644 LibOS/shim/test/ltp/ltp-bug-1075.cfg create mode 100644 LibOS/shim/test/ltp/ltp-bug-1248.cfg create mode 100644 LibOS/shim/test/ltp/ltp-sgx.cfg create mode 100644 LibOS/shim/test/ltp/ltp.cfg create mode 100644 LibOS/shim/test/ltp/makevars.mk create mode 100644 LibOS/shim/test/ltp/manifest.template create mode 100755 LibOS/shim/test/ltp/runltp_xml.py diff --git a/Examples/.gitignore b/Examples/.gitignore new file mode 100644 index 00000000..128c6a9c --- /dev/null +++ b/Examples/.gitignore @@ -0,0 +1,9 @@ +*.manifest +*.manifest.sgx +*.sig +*.token +pal_loader +*.o +*~ +*.swp +__pycache__ diff --git a/Examples/README.md b/Examples/README.md new file mode 100644 index 00000000..7b50e9ae --- /dev/null +++ b/Examples/README.md @@ -0,0 +1,66 @@ +# Graphene Applications + +This repository contains application samples for +[Graphene Library OS](https://github.com/oscarlab/graphene). +For how to build and run the Graphene Library OS, +please see the README in the Graphene repository. + +For instructions how to build and run each application under Graphene, +please see the README or README.md in each subdirectory. + +## How to Contribute? + +If you are interested in submitting an application sample for Graphene, +please submit a pull request to this GitHub repository. + +Please put your application sample in a subdirectory with a +comprehensible name. Ideally, the subdirectory name should be the same +as your application. In addition, your application sample should +have the following elements: + +- `README.md`: + Please document the tested environment and instructions for + building and running the application. If your application sample + has any known issues or requirements, please also specify them in + the documentation. + +- `Makefile`: + Users should be able to build your application sample by running + the `make` command. If your application needs extra building steps, + please document them in the `README.md`. In addition, we ask you + to provide sufficient comments in the `Makefile` to help users + understand the build process. If your application also runs on + Graphene-SGX, please include the commands for signing and retrieving + the token in the `Makefile`. + +- Manifest(s): + Please provide all the manifests needed for running your application + sample. Do not hard-code any user-specific path or personal info + in the manifests. The ideal way is to create manifest templates that + contain variables to be replaced by runtime options in `Makefile`. + See other subdirectories for examples of the manifest templates. + We also ask you to provide sufficient comments in all the manifests + to help users understand the environment. + +- Sample inputs and test suites: + If you have any inputs and test suites for testing the application, + please provide them in the same subdirectory, too. + +Please do not include any tarball of source code or binaries in the +application samples. If an application requires downloading the source +code or binaries, please provide instructions in the `README.md`, or +download them automatically and verify the checksums as part of the +build process. + +## Contact + +For any questions or bug reports, please send an email to + or report an issue in the following +GitHub repositories: + +- Graphene issues: +- Application sample issues (manifest, configuration, scripts): + + +Our mailing list is publicly archived +[here](https://groups.google.com/forum/#!forum/graphene-support). diff --git a/Examples/apache/.gitignore b/Examples/apache/.gitignore new file mode 100644 index 00000000..00ddab3d --- /dev/null +++ b/Examples/apache/.gitignore @@ -0,0 +1,7 @@ +/httpd-*/* +/*.tar.gz +/install/* +/result-* +/OUTPUT +/ssl/server.* +/ssl/ca.* diff --git a/Examples/apache/Makefile b/Examples/apache/Makefile new file mode 100644 index 00000000..91b68ce9 --- /dev/null +++ b/Examples/apache/Makefile @@ -0,0 +1,279 @@ +# Build the manifest for Apache (httpd): +# +# - make Building for Linux +# - make DEBUG=1 Building for Linux, with Graphene debug output +# - make SGX=1 Building for SGX +# - make SGX=1 DEBUG=1 Building for SGX +# +# Use `make clean` to remove Graphene-generated files. +# +# Use `make distclean` to further remove the Apache tarball, source code, +# and installation. + +THIS_DIR := $(dir $(lastword $(MAKEFILE_LIST))) + +INSTALL_DIR ?= $(THIS_DIR)install +HTTPD_SRC ?= $(THIS_DIR)httpd-2.4.41 +HTTPD_CHECKSUM ?= 3c0f9663240beb0f008acf3b4501c4f339d7467ee345a36c86c46b4d6f3a5461 + +# Mirros for downloading the Apache source code +HTTPD_MIRRORS ?= \ + https://www-eu.apache.org/dist/ \ + https://www-us.apache.org/dist/ \ + https://ftp.fau.de/apache/ \ + https://archive.apache.org/dist/ + +# Address and port for the Apache server to listen +LISTEN_HOST ?= 127.0.0.1 +LISTEN_PORT ?= 8001 + +# Relative path to Graphene root +GRAPHENEDIR ?= $(THIS_DIR)../../../../.. + +ifeq ($(DEBUG),1) +GRAPHENEDEBUG = inline +else +GRAPHENEDEBUG = none +endif + +.PHONY: all +all: $(INSTALL_DIR)/bin/httpd httpd.manifest pal_loader config testdata ssldata +ifeq ($(SGX),1) +all: httpd.manifest.sgx httpd.sig httpd.token +endif + +# The make targets for downloading and compiling the Apache source code, and +# installing the binaries. + +$(INSTALL_DIR)/bin/httpd: $(HTTPD_SRC)/configure + cd $(HTTPD_SRC) && ./configure --prefix=$(abspath $(INSTALL_DIR)) \ + --with-mpm=prefork --enable-mpms-shared='prefork worker event' \ + --enable-ssl + cd $(HTTPD_SRC) && $(MAKE) + cd $(HTTPD_SRC) && $(MAKE) install + +$(HTTPD_SRC)/configure: $(HTTPD_SRC).tar.gz + tar -mxzf $< + +$(HTTPD_SRC).tar.gz: + $(GRAPHENEDIR)/Scripts/download --output $@ --sha256 $(HTTPD_CHECKSUM) $(foreach mirror,$(HTTPD_MIRRORS),--url $(mirror)httpd/$(HTTPD_SRC).tar.gz) + +# Apache dependencies (generate from ldd): +# +# For SGX, the manifest needs to list all the libraries loaded during the +# execution, so that the signer can include the file checksums. +# +# The dependencies are generated from the ldd results. + +# We need to replace Glibc dependencies with Graphene-specific Glibc. The Glibc +# binaries are already listed in the manifest template, so we can skip them +# from the ldd results +GLIBC_DEPS = linux-vdso /lib64/ld-linux-x86-64 libc libm librt libdl libpthread + +$(INSTALL_DIR)/conf/httpd.conf: $(INSTALL_DIR) + +# Listing all the enabled modules in Apache, by greping httpd.conf +.INTERMEDIATE: httpd-modules +httpd-modules: $(INSTALL_DIR)/conf/httpd.conf + @grep "^LoadModule" $(INSTALL_DIR)/conf/httpd.conf | \ + awk '{print "$(INSTALL_DIR)/" $$3}' > $@ + @echo $(INSTALL_DIR)/modules/mod_mpm_worker.so >> $@ + @echo $(INSTALL_DIR)/modules/mod_mpm_event.so >> $@ + @echo $(INSTALL_DIR)/modules/mod_ssl.so >> $@ + +# Listing all the Apache dependencies, besides Glibc libraries +.INTERMEDIATE: httpd-ldd +httpd-ldd: httpd-modules + @for F in $(INSTALL_DIR)/bin/httpd `cat httpd-modules`; do \ + ldd $$F >> $@ || exit 1; done + +.INTERMEDIATE: httpd-deps +httpd-deps: httpd-ldd + @cat $< | awk '{if ($$2 =="=>") {split($$1,s,/\./); print s[1]}}' \ + | sort | uniq | grep -v -x $(patsubst %,-e %,$(GLIBC_DEPS)) > $@ + +# Generating manifest rules for Apache modules +.INTERMEDIATE: httpd-trusted-mods +httpd-trusted-mods: httpd-modules + @for F in `cat httpd-modules`; do \ + N=`echo $$F | awk '{n=split($$1,s,/\/|\./); print s[n-1]}'`; \ + echo -n "sgx.trusted_files.$$N = file:$$F\\\\n"; \ + done > $@ + +# Generating manifest rules for Apache dependencies +.INTERMEDIATE: httpd-trusted-libs +httpd-trusted-libs: httpd-deps httpd-modules + @HTTPD_LIBS="$(INSTALL_DIR)/bin/httpd `cat httpd-modules`" && \ + for F in `cat httpd-deps`; do \ + P=`ldd $$HTTPD_LIBS | grep $$F | awk '{print $$3; exit}'`; \ + N=`echo $$F | tr --delete '-'`; \ + echo -n "sgx.trusted_files.$$N = file:$$P\\\\n"; \ + done > $@ + +httpd.manifest: httpd.manifest.template httpd-trusted-mods httpd-trusted-libs + sed -e 's|$$(GRAPHENEDIR)|'"$(GRAPHENEDIR)"'|g' \ + -e 's|$$(GRAPHENEDEBUG)|'"$(GRAPHENEDEBUG)"'|g' \ + -e 's|$$(INSTALL_DIR)|'"$(INSTALL_DIR)"'|g' \ + -e 's|$$(INSTALL_DIR_ABSPATH)|'"$(abspath $(INSTALL_DIR))"'|g' \ + -e 's|$$(HTTPD_TRUSTED_MODS)|'"`cat httpd-trusted-mods`"'|g' \ + -e 's|$$(HTTPD_TRUSTED_LIBS)|'"`cat httpd-trusted-libs`"'|g' \ + $< > $@ + +# Generating the SGX-specific manifest (httpd.manifest.sgx), the enclave signature, +# and the token for enclave initialization. +httpd.manifest.sgx: httpd.manifest + $(GRAPHENEDIR)/Pal/src/host/Linux-SGX/signer/pal-sgx-sign \ + -libpal $(GRAPHENEDIR)/Runtime/libpal-Linux-SGX.so \ + -key $(GRAPHENEDIR)/Pal/src/host/Linux-SGX/signer/enclave-key.pem \ + -manifest $< -output $@ + +httpd.sig: httpd.manifest.sgx + +httpd.token: httpd.sig + $(GRAPHENEDIR)/Pal/src/host/Linux-SGX/signer/pal-sgx-get-token \ + -output httpd.token -sig httpd.sig + +# Extra executables +pal_loader: + ln -s $(GRAPHENEDIR)/Runtime/pal_loader $@ + +# Apache configuration and test data + +.PHONY: config +config: $(INSTALL_DIR)/conf/httpd-graphene.conf $(INSTALL_DIR)/conf/extra/httpd-ssl-graphene.conf + +$(INSTALL_DIR)/conf/httpd-graphene.conf: $(INSTALL_DIR)/conf/httpd.conf + sed -e "s|^Listen |#Listen |g" \ + -e "s|^User |#User |g" \ + -e "s|^Group |#Group |g" \ + -e "s|^LoadModule mpm_prefork|#LoadModule mpm_prefork|g" \ + -e "s|^#LoadModule ssl_module|LoadModule ssl_module|g" \ + -e "s|^#Include conf/extra/httpd-ssl.conf|Include conf/extra/httpd-ssl-graphene.conf|g" \ + -e "s|#EnableMMAP off|EnableMMAP off|g" \ + -e "s|#EnableSendfile on|EnableSendfile on|g" \ + $< > $@ + echo "\n\ +\n\ + StartServers 4\n\ + MinSpareServers 1\n\ + MaxSpareServers 4\n\ + MaxConnectionsPerChild 0\n\ +\n" >> $@ + echo "\n\ +\n\ + StartServers 1\n\ + MinSpareThreads 25\n\ + MaxSpareThreads 75\n\ + ThreadsPerChild 25\n\ +\n" >> $@ + +$(INSTALL_DIR)/conf/extra/httpd-ssl-graphene.conf: $(INSTALL_DIR)/conf/extra/httpd-ssl.conf + sed -e "s|^Listen 443|Listen 127.0.0.1:8443|g" \ + -e "s|^||g" \ + -e "s|^ServerName www.example.com:443|ServerName www.example.com:8443|g" \ + -e "s|^SSLSessionCache|#SSLSessionCache|g" \ + $< > $@ + +# HTTP docs: +# Generating random HTML files in $(INSTALL_DIR)/htdocs/random + +RANDOM_DIR = $(INSTALL_DIR)/htdocs/random +RANDOM_FILES = \ + $(foreach n,1 2 3 4 5 6 7 8 9 10,2K.$n.html) \ + $(foreach n,1 2 3 4 5,10K.$n.html) \ + $(foreach n,1 2 3 4 5,100K.$n.html) \ + $(foreach n,1 2 3,1M.$n.html) \ + $(foreach n,1 2 3,10M.$n.html) \ + $(foreach n,1 2 3,100.$n.html) + +TEST_DATA = $(addprefix $(RANDOM_DIR)/,$(RANDOM_FILES)) + +$(RANDOM_DIR)/%.html: + mkdir -p $(RANDOM_DIR) + dd if=/dev/urandom of=$@ count=1 bs=$(basename $(basename $(notdir $@))) + +.PHONY: testdata +testdata: $(TEST_DATA) + +# SSL data: key and x.509 self-signed certificate (to test SSL/TLS) + +$(INSTALL_DIR)/conf/server.crt: ssl/ca_config.conf + openssl genrsa -out ssl/ca.key 2048 + openssl req -x509 -new -nodes -key ssl/ca.key -sha256 -days 1024 -out ssl/ca.crt -config ssl/ca_config.conf + openssl genrsa -out ssl/server.key 2048 + openssl req -new -key ssl/server.key -out ssl/server.csr -config ssl/ca_config.conf + openssl x509 -req -days 360 -in ssl/server.csr -CA ssl/ca.crt -CAkey ssl/ca.key -CAcreateserial -out ssl/server.crt + cp -f ssl/* $(INSTALL_DIR)/conf/ + +.PHONY: ssldata +ssldata: $(INSTALL_DIR)/conf/server.crt + +# Commands for running Apache +# +# - make start-native-server: +# Start the Apache server natively in the foreground (using the Prefork MPM) +# +# - make start-graphene-server: +# Start the Apache server under Graphene in the foreground (using the Prefork MPM) +# +# - make start-native-multithreaded-server: +# Start the Apache server natively in the foreground (using the Worker MPM) +# +# - make start-graphene-multithreaded-server: +# Start the Apache server under Graphene in the foreground (using the Worker MPM) +# +# - make clean-server: +# Clean up the PID files generated by Apache + +.PHONY: clean-server +clean-server: + $(RM) $(INSTALL_DIR)/logs/httpd-$(LISTEN_HOST)-$(LISTEN_PORT).pid + +.PHONY: start-native-server +start-native-server: all clean-server + @echo "Listen on $(LISTEN_HOST):$(LISTEN_PORT)" + $(PREFIX) $(INSTALL_DIR)/bin/httpd -D FOREGROUND \ + -f conf/httpd-graphene.conf \ + -C "LoadModule mpm_prefork_module modules/mod_mpm_prefork.so" \ + -C "Listen $(LISTEN_HOST):$(LISTEN_PORT)" \ + -C "ServerName $(LISTEN_HOST)" \ + -C "PidFile logs/httpd-$(LISTEN_HOST)-$(LISTEN_PORT).pid" + +.PHONY: start-graphene-server +start-graphene-server: all clean-server + @echo "Listen on $(LISTEN_HOST):$(LISTEN_PORT)" + $(PREFIX) ./pal_loader httpd.manifest -D FOREGROUND \ + -f conf/httpd-graphene.conf \ + -C "LoadModule mpm_prefork_module modules/mod_mpm_prefork.so" \ + -C "Listen $(LISTEN_HOST):$(LISTEN_PORT)" \ + -C "ServerName $(LISTEN_HOST)" \ + -C "PidFile logs/httpd-$(LISTEN_HOST)-$(LISTEN_PORT).pid" + +.PHONY: start-native-multithreaded-server +start-native-multithreaded-server: all clean-server + @echo "Listen on $(LISTEN_HOST):$(LISTEN_PORT)" + $(PREFIX) $(INSTALL_DIR)/bin/httpd -D FOREGROUND \ + -f conf/httpd-graphene.conf \ + -C "LoadModule mpm_worker_module modules/mod_mpm_worker.so" \ + -C "Listen $(LISTEN_HOST):$(LISTEN_PORT)" \ + -C "ServerName $(LISTEN_HOST)" \ + -C "PidFile logs/httpd-$(LISTEN_HOST)-$(LISTEN_PORT).pid" + +.PHONY: start-graphene-multithreaded-server +start-graphene-multithreaded-server: all clean-server + @echo "Listen on $(LISTEN_HOST):$(LISTEN_PORT)" + $(PREFIX) ./pal_loader httpd.manifest -D FOREGROUND \ + -f conf/httpd-graphene.conf \ + -C "LoadModule mpm_worker_module modules/mod_mpm_worker.so" \ + -C "Listen $(LISTEN_HOST):$(LISTEN_PORT)" \ + -C "ServerName $(LISTEN_HOST)" \ + -C "PidFile logs/httpd-$(LISTEN_HOST)-$(LISTEN_PORT).pid" + +.PHONY: clean +clean: + $(RM) *.manifest *.manifest.sgx *.token *.sig pal_loader OUTPUT result-* httpd-ldd httpd-modules tmp + +.PHONY: distclean +distclean: clean + $(RM) -r $(HTTPD_SRC).tar.gz $(HTTPD_SRC) $(INSTALL_DIR) + $(RM) ssl/server.* ssl/ca.* diff --git a/Examples/apache/README.md b/Examples/apache/README.md new file mode 100644 index 00000000..6e336c9f --- /dev/null +++ b/Examples/apache/README.md @@ -0,0 +1,61 @@ +# Apache + +This directory contains the Makefile and the template manifest for the most +recent version of Apache web server (as of this writing, version 2.4.41). This +was tested on a machine with SGX v1 and Ubuntu 16.04. + +The Makefile and the template manifest contain extensive comments. Please review +them to understand the requirements for Apache running under Graphene-SGX. + +We build Apache from the source code instead of using an existing installation. +On Ubuntu 16.04, please make sure that the following packages are installed: +```sh +sudo apt-get install -y build-essential flex libapr1-dev libaprutil1-dev libpcre2-dev \ + apache2-utils libssl-dev +``` + +# Quick Start + +```sh +# build Apache and the final manifest +make SGX=1 + +# run original Apache against HTTP and HTTPS benchmarks (benchmark-http.sh, uses ab) +make start-native-server & +./benchmark-http.sh 127.0.0.1:8001 +./benchmark-http.sh https://127.0.0.1:8443 +kill -SIGINT %% + +# run Apache in non-SGX Graphene against HTTP and HTTPS benchmarks +make start-graphene-server & +./benchmark-http.sh 127.0.0.1:8001 +./benchmark-http.sh https://127.0.0.1:8443 +kill -SIGINT %% + +# run Apache in Graphene-SGX against HTTP and HTTPS benchmarks +SGX=1 make start-graphene-server & +./benchmark-http.sh 127.0.0.1:8001 +./benchmark-http.sh https://127.0.0.1:8443 +kill -SIGINT %% + +# you can also test the server using other utilities like wget +wget http://127.0.0.1:8001/random/10K.1.html +wget https://127.0.0.1:8443/random/10K.1.html +``` + +# Running Apache with Different MPMs + +The Apache server can run with several different multi-processing modules +(MPMs). The two popular ones are *Prefork* and *Worker* MPMs. The Prefork +MPM uses multiple child processes with one thread each, and each process +handles one connection at a time. The Worker MPM uses multiple child processes +with many threads each, and each thread handles one connection at a time. + +The supplied Makefile allows to run Apache in both configurations: +```sh +make start-native-server # run with Prefork MPM +make start-graphene-server # run with Prefork MPM + +make start-native-multithreaded-server # run with Worker MPM +make start-graphene-multithreaded-server # run with Worker MPM +``` diff --git a/Examples/apache/benchmark-http.sh b/Examples/apache/benchmark-http.sh new file mode 120000 index 00000000..38087f85 --- /dev/null +++ b/Examples/apache/benchmark-http.sh @@ -0,0 +1 @@ +../common_tools/benchmark-http.sh \ No newline at end of file diff --git a/Examples/apache/httpd.manifest.template b/Examples/apache/httpd.manifest.template new file mode 100644 index 00000000..acb1ff4d --- /dev/null +++ b/Examples/apache/httpd.manifest.template @@ -0,0 +1,114 @@ +# Apache manifest example +# +# This manifest was prepared and tested on Ubuntu 16.04. +# +# Apache must be run with the pal_loader: +# +# ./pal_loader httpd.manifest