Merge pull request #515 from vcaputo/stage1_doc

doc: stage1 implementor's guide
2026-09-04 12:51:39 +00:00 · 2015-02-12 11:44:04 -08:00
parent 68a0c0b656 23b72d870d
commit c1bc4628d2
1 changed files with 101 additions and 0 deletions
@@ -0,0 +1,101 @@
+Stage1 ACI implementor's guide
+=============================
+
+Background
+----------
+
+Rocket's execution of containers is divided roughly into three separate stages:
+
+0. Stage 0: discovering, fetching, verifying, storing, and compositing of both application (stage 2) and stage 1 images for execution.
+1. Stage 1: execution of the stage 1 image from within the composite image prepared by stage 0.
+2. Stage 2: execution of individual application images within the containment afforded by stage 1.
+
+This separation of concerns is reflected in the file-system and layout of the composite image prepared by stage 0:
+
+0. Stage 0: `rkt` executable, and the Container Runtime Manifest created at "/var/lib/rkt/containers/$uuid/container".
+1. Stage 1: "stage1.aci", made available at "/var/lib/rkt/containers/$uuid/stage1" by `rkt run`.
+2. Stage 2: "$app.aci", made available at "/var/lib/rkt/containers/$uuid/stage1/rootfs/opt/stage2/$imageid" by `rkt run`.
+
+The stage 1 implementation is what creates the execution environment for the contained applications.  This occurs via entrypoints from stage 0 on behalf of `rkt run` and `rkt enter`.  These entrypoints are nothing more than executable programs located via Annotations from within the stage 1 ACI manifest, and executed from within the stage 1 of a given container at "/var/lib/rkt/containers/$uuid/stage1/rootfs".
+
+Stage 2 is the destination application images and stage 1 is the vehicle for getting us there from stage 0.  For any given container instance, the stage 1 may be completely different, allowing for flexibility in containment strategies employed within the same host while utilizing reusable application ACIs.
+
+Entrypoints
+-----------
+
+### `rkt run` => "coreos.com/rocket/stage1/run"
+
+0. rkt prepares the container's stage 1 and stage 2 images and Container Runtime Manifest under "/var/lib/rkt/containers/$uuid", acquiring an exclusive advisory lock on the directory.
+1. chdirs to "/var/lib/rkt/containers/$uuid"
+2. resolves the "coreos.com/rocket/stage1/run" entrypoint via Annotations found within "/var/lib/rkt/containers/$uuid/stage1/manifest"
+3. executes the resolved entrypoint relative to "/var/lib/rkt/containers/$uuid/stage1/rootfs"
+
+It is the responsibility of this entrypoint to consume the Container Runtime Manifest and execute the constituent apps in the appropriate environments as specified by the Container Runtime Manifest.
+
+The environment variable "RKT_LOCK_FD" contains the file descriptor number of the open directory handle for "/var/lib/rkt/containers/$uuid".  It is necessary that stage 1 leave this file descriptor open and in its locked state for the duration of the `rkt run`.
+
+In the bundled rocket stage 1 which includes systemd-nspawn and systemd, the entrypoint is a static Go program found at "/init" within the stage 1 ACI rootfs.  The majority of its execution entails generating a systemd-nspawn argument list and writing systemd unit files for the constituent apps before executing systemd-nspawn.  Systemd-nspawn then boots the stage 1 systemd with the just-written unit files for launching the contained apps.  The "/init" program is essentially a Container Runtime Manifest to systemd-nspawn + systemd.service translator.
+
+An alternative stage 1 could forego systemd-nspawn and systemd altogether, or retain these and introduce something like novm or qemu-kvm for greater isolation by first starting a VM.  All that is required is an executable at the place indicated by the "coreos.com/rocket/stage1/run" entrypoint which knows how to apply the Container Runtime Manifest and prepared ACI file-systems to good effect.
+
+
+#### Arguments
+* --debug to activate debugging
+* --private-net to trigger the creation of a private network
+
+
+### `rkt enter` => "coreos.com/rocket/stage1/enter"
+
+0. rkt verifies the container and image to enter are valid and running
+1. chdirs to "/var/lib/rkt/containers/$uuid"
+2. resolves the "coreos.com/rocket/stage1/enter" entrypoint via Annotations found within "/var/lib/rkt/containers/$uuid/stage1/manifest"
+3. executes the resolved entrypoint relative to "/var/lib/rkt/containers/$uuid/stage1/rootfs"
+
+In the bundled rocket stage 1 the entrypoint is a statically-linked C program found at "/enter" within the stage 1 ACI rootfs.  This program enters the namespaces of the systemd-nspawn container's PID 1 before executing the "/diagexec" program which then chroots into the specific application's rootfs loading the application's environment variables as well.
+
+An alternative stage 1 would need to do whatever is appropriate for entering the application's environment created by its own "coreos.com/rocket/stage1/run" entrypoint.
+
+#### Arguments
+
+1. image id of the specific application to enter
+2. cmd to execute
+3. any cmd arguments
+
+
+
+Examples
+--------
+
+### Stage1 ACI manifest
+
+```
+{
+    "acKind": "ImageManifest",
+    "acVersion": "0.2.0",
+    "name": "foo.com/rocket/stage1",
+    "labels": [
+        {   
+            "name": "version",
+            "value": "0.0.1"
+        },
+        {   
+            "name": "arch",
+            "value": "amd64"
+        },
+        {   
+            "name": "os",
+            "value": "linux"
+        }
+    ],
+    "annotations": [
+        {   
+            "name": "coreos.com/rocket/stage1/run",
+            "value": "/ex/run"
+        },
+        {   
+            "name": "coreos.com/rocket/stage1/enter",
+            "value": "/ex/enter"
+        }
+    ]
+}
+```