clearlinux/rkt
2
0
Fork 0
mirror of https://github.com/clearlinux/rkt.git synced 2026-09-01 11:26:05 +00:00
Code Issues Packages Projects Releases Wiki Activity

app-container: SPEC: format fixups

Browse Source
This commit is contained in:
Brandon Philips
2014-11-26 18:15:19 -08:00
parent a8ab53a795
commit d96cb05f69
1 changed files with 95 additions and 105 deletions
Download Patch File Download Diff File Expand all files Collapse all files
app-container/SPEC.md
+95 -105
View File
@@ -8,7 +8,7 @@ The "App Container" defines an image format, image discovery mechanism and execu
* Use common technologies for crypto, archive, compression and transport
* Use the DNS namespace to name and discover container images
To achieve these goals this specification is split out into a number of smaller sections.
To achieve these goals this specification is split out into a number of smaller sections.
1. The **[App Container Image](#app-conatiner-image)** defines: how files are assembled together into a single image, verified on download and placed onto disk to be run.
@@ -23,26 +23,36 @@ To achieve these goals this specification is split out into a number of smaller
To provide context to the specs outlined below we will walk through an example.
A user wants to launch a container running two processes. The two processes the user wants to run are the apps named `example.com/reduce-worker-register-1.0.0` and `example.com/reduce-worker-1.0.0`. First, the executor will check the cache and find that it doesn't have images available for these apps. So, it will make an HTTPS request to example.com and using the <meta> tags there finds that the containers can be found at:
A user wants to launch a container running two processes.
The two processes the user wants to run are the apps named `example.com/reduce-worker-register-1.0.0` and `example.com/reduce-worker-1.0.0`.
First, the executor will check the cache and find that it doesn't have images available for these apps.
So, it will make an HTTPS request to example.com and using the <meta> tags there finds that the containers can be found at:
https://storage-mirror.example.com/reduce-worker-register-1.0.0.aci
https://storage-mirror.example.com/reduce-worker-1.0.0.aci
The executor downloads these two "fileset images" and puts them into its local on-disk cache. Then the executor extracts two fresh copies of these filesets to create instances of the "on-disk app format" and reads the two app manifests to figure out what binaries will need to executed. It finds that the register app has a main entry point at /usr/bin/register and the worker has one at /usr/bin/reduce-worker.
The executor downloads these two "fileset images" and puts them into its local on-disk cache.
Then the executor extracts two fresh copies of these filesets to create instances of the "on-disk app format" and reads the two app manifests to figure out what binaries will need to executed.
It finds that the register app has a main entry point at /usr/bin/register and the worker has one at /usr/bin/reduce-worker.
Based on user input the executor now sets up the necessary cgroups, network interfaces, etc and forks the register and reduce-worker processes in their shared namespaces inside of the container.
At some point the container will get some notification that it needs to stop. The executor will send SIGTERM to the processes and after they have exited the postStop event handlers for each app will run. That will end the life of the container.
At some point the container will get some notification that it needs to stop.
The executor will send SIGTERM to the processes and after they have exited the postStop event handlers for each app will run.
That will end the life of the container.
Now, lets dive into the pieces that took us from two URLs to a running container on our system.
## App Container Image
An *App Container Image* (ACI) contains all files and metadata needed to execute a given app. In some ways you can think of an ACI as equivalent to a static binary. This file layout must be followed for the app to be executed by an Executor.
An *App Container Image* (ACI) contains all files and metadata needed to execute a given app.
In some ways you can think of an ACI as equivalent to a static binary.
This file layout must be followed for the app to be executed by an Executor.
### Image Layout
The on-disk layout of an app container is straightforward. It includes a *rootfs* with all of the files that will exist in the root of the app and an *app manifest* describing how to execute the app.
The on-disk layout of an app container is straightforward.
It includes a *rootfs* with all of the files that will exist in the root of the app and an *app manifest* describing how to execute the app.
```
/app
@@ -53,9 +63,12 @@ The on-disk layout of an app container is straightforward. It includes a *rootfs
### Image Archives
The ACI archive format aims for flexibility and relies on very boring technologies: HTTP, gpg, tar and gzip. This set of formats makes it easy to build, host and secure a container using technologies that are battle tested.
The ACI archive format aims for flexibility and relies on very boring technologies: HTTP, gpg, tar and gzip.
This set of formats makes it easy to build, host and secure a container using technologies that are battle tested.
Images achives MUST be a tar formatted file. The image may be optionally compressed with gzip, bzip2 or xz. After compression images may also be encrypted with AES symmetric encryption. Simple and standard UNIX commands can take the on-disk layout and transform it into a signed aci:
Images achives MUST be a tar formatted file.
The image may be optionally compressed with gzip, bzip2 or xz. After compression images may also be encrypted with AES symmetric encryption.
Simple and standard UNIX commands can take the on-disk layout and transform it into a signed aci:
```
tar cvvf reduce-worker.tar app rootfs
@@ -76,45 +89,44 @@ The default digest format is sha256 but all hash IDs in this format are prefixed
echo sha256-$(sha256sum reduce-worker.tar |awk ‘{print $1}’)
**Note**: that the key distribution mechanism is not defined here. Implementations of the app container spec will need to provide a mechanism for users to configure the list of signing keys to trust or use the key discovery described in "App Container Image Discovery".
**Note**: that the key distribution mechanism is not defined here.
Implementations of the app container spec will need to provide a mechanism for users to configure the list of signing keys to trust or use the key discovery described in "App Container Image Discovery".
Example container builder for deterministic Go transport containers: **TODO** link to actool
### App Manifest
The [app manifest](#heading=h.y4wfi6t6yhu3) is a JSON file that includes all of the details to execute a main process from the rootfs. Execution details include mount points that should exist, the user, the command args, default cgroup settings and more. The manifest also defines binaries to execute in response to lifecycle events of the main process such as *preStart* and *postStop*.
The [app manifest](#heading=h.y4wfi6t6yhu3) is a JSON file that includes all of the details to execute a main process from the rootfs.
Execution details include mount points that should exist, the user, the command args, default cgroup settings and more.
The manifest also defines binaries to execute in response to lifecycle events of the main process such as *preStart* and *postStop*.
Image Format TODO
* Define the garbage collection lifecycle of the container filesystem including:
* Format of app exit code and signal
* The refcounting plan for resources consumed by the ACE such as volumes
* Define the lifecycle of the container as all exit or first to exit
* Define security requirements for a container. In particular is any isolation of users required between containers? What user does each application run under and can this be root (i.e. "real" root in the host).
* Define how apps are supposed to communicate; can they/do they ‘see’ each other (a section in the apps perspective would help)?
## App Container Executor
App Containers are a combination of a number of technologies which are not aware of each other. This specification attempts to define a reasonable subset of steps to accomplish a few goals:
App Containers are a combination of a number of technologies which are not aware of each other.
This specification attempts to define a reasonable subset of steps to accomplish a few goals:
* Creating a filesystem hierarchy in which the app will execute
* Running the app process inside of a combination of resource and namespace isolations
* Executing the application inside of this environment
There are two "perspectives" in this process. The "*executor*" perspective consists of the steps that the container executor must take to set up the containers. The "*app*" perspective is how the app processes inside the container see the environment.
There are two "perspectives" in this process.
The "*executor*" perspective consists of the steps that the container executor must take to set up the containers. The "*app*" perspective is how the app processes inside the container see the environment.
### Executor Perspective
#### Filesystem Setup
Every execution of an app container should start from a clean copy of the app fileset. The simplest implementation will take a app fileset image and extract it into a new directory:
Every execution of an app container should start from a clean copy of the app fileset.
The simplest implementation will take a app fileset image and extract it into a new directory:
```
cd $(mktemp -d -t temp.XXXX)
@@ -122,66 +134,70 @@ mkdir hello
tar xzvf /var/lib/pce/hello.aci -C hello
```
Other implementations could increase performance and de-duplicate data by building on top of overlay filesystems, copy-on-write block devices, or a content-addressed file store. These details are orthogonal to the runtime environment.
Other implementations could increase performance and de-duplicate data by building on top of overlay filesystems, copy-on-write block devices, or a content-addressed file store.
These details are orthogonal to the runtime environment.
#### Container Runtime Manifest
A container executes one or more apps with shared PID namespace, network namespace, mount namespace, IPC namespace and UTS namespace. Each app will start pivoted (i.e. chrooted) into its own unique read-write rootfs before execution. The definition of the container is a list of apps that should be launched together. This is codified in a [Container Runtime Manifest](#heading=h.j03f37q4hupc).
A container executes one or more apps with shared PID namespace, network namespace, mount namespace, IPC namespace and UTS namespace.
Each app will start pivoted (i.e. chrooted) into its own unique read-write rootfs before execution. The definition of the container is a list of apps that should be launched together.
This is codified in a [Container Runtime Manifest](#heading=h.j03f37q4hupc).
This example container will use a set of three apps with the given names and manifest hashes:
```
example.com/reduce-worker-1.0.0 sha256-277205b3ae3eb3a8e042a62ae46934b470e431ac
example.com/worker-backup-1.0.0 sha256-3e86b59982e49066c5d813af1c2e2579cbf573de
example.com/reduce-worker-register-1.0.0 sha256-86298e1fdb95ec9a45b5935504e26ec29b8feffa
```
#### Volume Setup
Volumes that are specified in the Container Runtime Manifest are mounted into each of the apps via a bind mount. For example say that the worker-backup and reduce-worker both have a MountPoint named "work". Then the container manager will bind mount the host's /opt/tenant1/database directory into the Path of each of the matching "work" MountPoints of the two containers.
Volumes that are specified in the Container Runtime Manifest are mounted into each of the apps via a bind mount.
For example say that the worker-backup and reduce-worker both have a MountPoint named "work".
Then the container manager will bind mount the host's /opt/tenant1/database directory into the Path of each of the matching "work" MountPoints of the two containers.
#### Network Setup
An App Container must have a [layer 3](http://en.wikipedia.org/wiki/Network_layer) (commonly called the IP layer) network interface; this can be instantiated in any number of ways (e.g. veth, macvlan, ipvlan, device pass-through). The network interface should be configured with an IPv4/IPv6 address that is reachable from other containers.
An App Container must have a [layer 3](http://en.wikipedia.org/wiki/Network_layer) (commonly called the IP layer) network interface; this can be instantiated in any number of ways (e.g. veth, macvlan, ipvlan, device pass-through).
The network interface should be configured with an IPv4/IPv6 address that is reachable from other containers.
#### Logging
Apps should log to stdout and stderr. The container manager is responsible for capturing and persisting the output.
Apps should log to stdout and stderr. The container manager is responsible for capturing and persisting the output.
If the application detects other logging options, such as the /run/systemd/system/journal socket, it may optionally upgrade to using those mechanisms. Note that logging mechanisms other than stdout and stderr are not required by this specification (or tested by the the compliance tests).
If the application detects other logging options, such as the /run/systemd/system/journal socket, it may optionally upgrade to using those mechanisms.
Note that logging mechanisms other than stdout and stderr are not required by this specification (or tested by the the compliance tests).
### Apps Perspective
#### Execution Environment
* **Working directory** always the root of the application image
* **PATH** /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
* **USER, LOGNAME** username of the user executing this app
* **HOME** home directory of the user
* **SHELL** login shell of the user
* **AC_APP_NAME** the entrypoint that this process was defined from
* **AC_METADATA_URL** URL that the metadata service for this container can be found
### Isolators
Isolators enforce resource constraints rather than namespacing. Isolators may be applied to individual applications, to whole containers, or to both. Some well known isolators can be verified by the specification. Additional isolators will be added to this specification over time.
Isolators enforce resource constraints rather than namespacing.
Isolators may be applied to individual applications, to whole containers, or to both.
Some well known isolators can be verified by the specification.
Additional isolators will be added to this specification over time.
|Name|Type| Schema |Example|
|----|----|--------|-------|
|cpu/shares/ |string|"&lt;uint&gt;" |"4096"|
|Name|Type|Schema|Example|
|-------------------------|------|------------------------------------|----------------|
|cpu/shares/ |string|"&lt;uint&gt;" |"4096" |
|memory/limit |string|"&lt;bytes&gt;" |"1G", "5T", "4K"|
|blockIO/readBandwidth |string|"&lt;path to file&gt; &lt;bytes&gt;"|"/tmp 1K"|
|blockIO/writeBandwidth |string|"&lt;path to file&gt; &lt;bytes&gt;"|"/tmp 1K"|
|networkIO/readBandwidth |string|"&lt;device name&gt; &lt;bytes&gt;" |"eth0 100M"|
|networkIO/writeBandwidth |string|"&lt;device name&gt; &lt;bytes&gt;" |"eth0 100M"|
|privateNetwork |string|"&lt;true&#124;false&gt;" |"true"|
|capabilities/boundingSet |string|"&lt;cap&gt; &lt;cap&gt; ..." |"CAP_NET_BIND_SERVICE CAP_SYS_ADMIN"|
|blockIO/readBandwidth |string|"&lt;path to file&gt; &lt;bytes&gt;"|"/tmp 1K" |
|blockIO/writeBandwidth |string|"&lt;path to file&gt; &lt;bytes&gt;"|"/tmp 1K" |
|networkIO/readBandwidth |string|"&lt;device name&gt; &lt;bytes&gt;" |"eth0 100M" |
|networkIO/writeBandwidth |string|"&lt;device name&gt; &lt;bytes&gt;" |"eth0 100M" |
|privateNetwork |string|"&lt;true&#124;false&gt;" |"true" |
|capabilities/boundingSet |string|"&lt;cap&gt; &lt;cap&gt; ..." |"CAP_NET_BIND_SERVICE CAP_SYS_ADMIN"|
#### Types
@@ -190,15 +206,19 @@ Isolators enforce resource constraints rather than namespacing. Isolators may be
## App Container Image Discovery
An app name has a URL like structure, for example `example.com/reduce-worker-1.0.0`. However, there is no scheme on this app name so we can't resolve it to a fileset image.
An app name has a URL like structure, for example `example.com/reduce-worker-1.0.0`.
However, there is no scheme on this app name so we can't resolve it to a fileset image.
### Simple Download
First, try to fetch the app fileset image by prepending https:// and appending .aci. If this fails, move on to meta discovery. If this succeeds, try fetching the signature using the .sig extension.
First, try to fetch the app fileset image by prepending https:// and appending .aci.
If this fails, move on to meta discovery.
If this succeeds, try fetching the signature using the .sig extension.
### Meta Discovery
If simple discovery fails then we use HTTPS+HTML meta tags to resolve an app name to a downloadable URL. For example if the ACE is looking for `example.com/reduce-worker-1.0.0` it will request:
If simple discovery fails then we use HTTPS+HTML meta tags to resolve an app name to a downloadable URL.
For example if the ACE is looking for `example.com/reduce-worker-1.0.0` it will request:
https://example.com/reduce-worker?ac-discovery=1
@@ -228,15 +248,16 @@ curl $(echo "$urltmpl" | sed "s/{name}/$appname/")
In our example above this would be:
```
sig: https://storage.example.com/linux/amd64/reduce-worker-1.0.0.sig
aci: https://storage.example.com/linux/amd64/reduce-worker-1.0.0.aci
keys: https://example.com/pubkeys.gpg
```
This mechanism is only used for discovery of contents URLs. Anything implementing this spec should enforce any signing rules set in place by the operator and ensure the app manifest provided by the fetched app fileset image are all prefixed from the same domain.
This mechanism is only used for discovery of contents URLs.
Anything implementing this spec should enforce any signing rules set in place by the operator and ensure the app manifest provided by the fetched app fileset image are all prefixed from the same domain.
Discovery URLs that require interpolation are RFC 6570 URI templates that have a single variable ‘{name}’.
Discovery URLs that require interpolation are RFC 6570 URI templates.
Inspired by: https://golang.org/cmd/go/#hdr-Remote_import_paths
@@ -245,20 +266,16 @@ Inspired by: https://golang.org/cmd/go/#hdr-Remote_import_paths
For a variety of reasons, it is desirable to not write files to the filesystem in order to run a container:
* Environment variables are not portable to virtual machines
* Config-drives make assumptions about filesystems
* Secrets can be kept outside of the container (such as the identity endpoint specified below)
* Writing files leads to assumptions like a libc environment attempting parse /etc/hosts
* The container can be run on top of a cryptographically secured read-only filesystem
* Metadata is a proven system for virtual machines
### Metadata Server
The ACE must provide a Metadata server on the address given to the container via AC_METADATA_URL. By convention the default will be http://169.254.169.254.
The ACE must provide a Metadata server on the address given to the container via AC_METADATA_URL.
By convention the default will be http://169.254.169.254.
Clients querying any of these endpoints must specify the Metadata-Flavor: AppContainer header.
@@ -272,11 +289,12 @@ Retrievable at `http://169.254.169.254/acMetadata/v1/container`
|---------------------------|
|annotations/ | A directory of metadata values passed to the container.|
|manifest | The container manifest JSON |
|uid |The unique execution container uid.|
|uid | The unique execution container uid.|
### App Metadata
Every running process will be able to introspect its App Name via the AC_APP_NAME environment variable. This is necessary in order to query for the correct endpoint metadata.
Every running process will be able to introspect its App Name via the AC_APP_NAME environment variable.
This is necessary to query for the correct endpoint metadata.
Retrievable at `http://169.254.169.254/acMetadata/v1/apps/${ac_app_name}/`
@@ -284,11 +302,12 @@ Retrievable at `http://169.254.169.254/acMetadata/v1/apps/${ac_app_name}/`
|-----------------------------|
|annotations/ | A directory of metadata values on the entrypoint manifest.|
|image/manifest | The original manifest file of the app. |
|image/id |Cryptographic image ID this app is on.|
|image/id | Cryptographic image ID this app is on.|
### Identity Endpoint
As a basic building block for building a secure identity system, the metadata service must provide an HMAC (described in [RFC2104](https://www.ietf.org/rfc/rfc2104.txt)) endpoint for use by the apps in the container. This gives a cryptographically verifiable identity to the container based on its container unique ID and the container HMAC key, which is held securely by the ACE.
As a basic building block for building a secure identity system, the metadata service must provide an HMAC (described in [RFC2104](https://www.ietf.org/rfc/rfc2104.txt)) endpoint for use by the apps in the container.
This gives a cryptographically verifiable identity to the container based on its container unique ID and the container HMAC key, which is held securely by the ACE.
Accessible at `http://169.254.169.254/acMetadata/v1/container/hmac`
@@ -300,7 +319,7 @@ Accessible at `http://169.254.169.254/acMetadata/v1/container/hmac`
## AC Name Type
An AC Name Type is restricted to lowercase characters accepted by the DNS [RFC](http://tools.ietf.org/html/rfc1123#page-13) and "/".
An AC Name Type is restricted to lowercase characters accepted by the DNS [RFC](http://tools.ietf.org/html/rfc1123#page-13) and "/".
Examples:
@@ -309,11 +328,13 @@ Examples:
* example.com/ourapp-1.0.0
* sub-domain.example.com/org/product/release-1.0.0
The AC Name Type is used as the primary key for a number of fields in the schemas below. The schema validator will ensure that the keys conform to these constraints.
The AC Name Type is used as the primary key for a number of fields in the schemas below.
The schema validator will ensure that the keys conform to these constraints.
## AC Filesets
The "app container image" MAY contain a second optional manifest to describe how to assemble the final rootfs from a collection of other images. These other images may not container an app manifest in which case they are simply a "fileset".
The "app container image" MAY contain a second optional manifest to describe how to assemble the final rootfs from a collection of other images.
These other images may not container an app manifest in which case they are simply a "fileset".
```
/fileset
@@ -322,10 +343,16 @@ The "app container image" MAY contain a second optional manifest to describe how
/rootfs/usr/bin/reduce-worker
```
As an example you might have an that needs special certificates layered into its filesystem. In this case you can reference the name "example.com/trusted-certificate-authority-1.0.0" as a dependency in the fileset manifest. The dependencies are applied in order and each fileset can clobber files from the previous fileset. Optionally, filesets can be applied to a subtree, such as `/etc/ssl` for the trusted-cretificate-authority-1.0.0.
As an example you might have an that needs special certificates layered into its filesystem.
In this case you can reference the name "example.com/trusted-certificate-authority-1.0.0" as a dependency in the fileset manifest.
The dependencies are applied in order and each fileset can clobber files from the previous fileset.
Optionally, filesets can be applied to a subtree, such as `/etc/ssl` for the trusted-cretificate-authority-1.0.0.
## Manifest Schemas
## App Manifest Schema
### App Manifest Schema
JSON Schema for the App Image Manifest
@@ -408,46 +435,27 @@ JSON Schema for the App Image Manifest
```
* **acVersion** is required and represents the version of the schema specification that the manifest implements (string, must be in [semver](http://semver.org/) format)
* **acKind** is required and must be set to "ImageManifest"
* **name** should be unique for every build of an app. It will be used as a human readable index to the container image. The name is restricted to the AC Name formatting.
* **os** is required (string; currently, the only supported value is "linux"). Together with “Arch”, this can be considered to describe the syscall ABI this image requires.
* **arch** is required (string; currently, the only supported value is "amd64"). Together with “OS”, this can be considered to describe the syscall ABI this image requires.
* **exec** the executable to launch and any flags (array of strings) (ACE can append or override)
* **user/group** are required, and indicate either the GID/UID or the username/group name the app should run as inside of the container (freeform string). If the user or group field begins with a "/" the owner and group of the file found at that absolute path is used as the GID/UID of the process.
* **eventHandlers** are optional, and should be a list of eventHandler objects. eventHandlers allow the app to have several hooks based on lifecycle events. For example, you may want to execute a script before the main process starts up to download a dataset or backup onto the filesystem. An eventHandler is a simple object with two fields - an **exec** (array of strings, ACE can append or override), and a **name**, which should be one of:
* **pre-start** - will be executed and must exit before the long running main **exec** binary is launched
* **post-stop** - if the main **exec** process is killed then this is ran. This can be used to cleanup resources in the case of clean application shutdown, but cannot be relied upon in the face of machine failure.stopped
* **environment** the app’s preferred environment variables (map of freeform strings) (ACE can append)
* **mountPoints** are the locations where a container is expecting external data to mounted. The name indicates an executor-defined label to look up a mount point, and the path stipulates where it should actually be mounted inside the rootfs. The key is restricted to the AC Name Type formatting.
* **ports** are the protocols and port numbers that the container will be listening on once started. The key is restricted to the AC Name formatting. This information is primarily informational to help the user find ports that are not well known. It could also optionally be used to limit the inbound connections to the container via firewall rules to only ports that are explicitly exposed.
* **socketActivated** if this is set to true then the application expects to be [socket activated](http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html) on these ports. The ACE must pass file descriptors using the [socket activation protocol](http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html)[ ](http://www.freedesktop.org/software/systemd/man/sd_listen_fds.html)that are listening on these ports when starting this container. If multiple apps in the same container are using socket activation then the ACE must match the sockets to the correct apps using getsockopt() and getsockname().
* **isolators** is a list of well-known and optional isolation steps that should be applied to the app. **name** is restricted to the AC Name formatting and **val** can be a freeform string. Any isolators specified in the App Manifest can be overridden at runtime via the Container Runtime Manifest. The executor can either ignore isolator keys it does not understand or error. In practice this means there might be certain isolators (for example, an AppArmor policy) that an executor doesn't understand so it will simply skip that entry.
* **annotations** key/value store that can be used by systems outside of the ACE (ACE can override). The key is restricted to the [AC Name](https://docs.google.com/a/coreos.com/document/d/1qaUwBhKnWSdkPtw2-NhoN4XIyiz0Nd3_xsh_u-71fR0/edit#heading=h.5538mwoeejrw) formatting. If you are defining new annotations, please consider submitting them to the specification. If you intend for your field to remain special to your application please be a good citizen and prefix an appropriate namespace to your key names. Recognized annotations include:
* **created** is the date on which this container was built (string, must be in [RFC3339](https://www.ietf.org/rfc/rfc3339.txt) format)
* **authors** contact details of the people or organization responsible for the containers (list of strings)
* **homepage** URL to find more information on the container (string, must be a URL with scheme HTTP or HTTPS)
* **documentation** URL to get documentation on this container (string, must be a URL with scheme HTTP or HTTPS)
## Container Runtime Manifest Schema
### Container Runtime Manifest Schema
```
{
@@ -506,32 +514,20 @@ JSON Schema for the App Image Manifest
```
* **acVersion** is required and represents the version of the schema spec (string, must be in [semver](http://semver.org/) format)
* **acKind** is required and must be set to "ContainerRuntimeManifest"
* **uuid** an [RFC4122 UUID](http://www.ietf.org/rfc/rfc4122.txt) that represents this instance of the container (string, must be in [RFC4122](http://www.ietf.org/rfc/rfc4122.txt) format)
* **apps** the list of apps that will execute inside of this container
* **app** the name of the app (string, restricted to AC Name formatting)
* **imageID** the content hash of the image that this app will execute inside of (string, must be of the format "type-value", where “type” is “sha256” and value is the hex encoded string of the hash)
* **isolators** the list of isolators that should be applied to this app (key is restricted to the AC Name formatting and the value can be a freeform string)
* **annotations** arbitrary metadata appended to the app (key is restricted to the AC Name formatting and the value can be a freeform string)
* **volumes** the list of volumes which should be mounted into each application’s filesystem
* **kind** string, currently either "empty" or “host” (bind mount)
* **fulfills** the MountPoints of the containers that this volume can fulfill (string, restricted to AC Name formatting)
* **isolators** the list of isolators that will apply to all apps in this container (name is restricted to the AC Name formatting and the value can be a freeform string)
* **annotations** arbitrary metadata the executor should make available to applications via the metadata service (key is restricted to the AC Name formatting and the value can be a freeform string)
## FileSet Manifest
### FileSet Manifest
```
{
@@ -565,15 +561,9 @@ JSON Schema for the App Image Manifest
```
* **acVersion** is required and represents the version of the schema spec (string, must be in [semver](http://semver.org/) format)
* **acKind** is required and must be set to "FileSetManifest"
* **dependencies** list of filesets that need to be placed down into the rootfs before the files from this fileset. The ordering is significant.
* **hash** content hash of the fileset
* **name** name of the app
* **root** the place in this filesets hierarchy that the dependent rootfs should be placed
* **files** ordered list of files that exist in this fileset. Ordering matters as it is the order items should be added to the tar file for hashing.