mirror of
https://github.com/clearlinux/rkt.git
synced 2026-09-06 22:01:54 +00:00
Merge pull request #103 from jonboulle/master
appc: add README, various actool fixes
This commit is contained in:
Jonathan Boulle
+175
-7
@@ -1,13 +1,181 @@
|
||||
# app container format
|
||||
# App Container
|
||||
|
||||
This repository contains schema definitions and tools for the App Container specifications.
|
||||
See [SPEC.md](SPEC.md) for details of the specifications themselves.
|
||||
## Overview
|
||||
|
||||
This repository contains schema definitions and tools for the App Container specification.
|
||||
See [SPEC.md](SPEC.md) for details of the specification itself.
|
||||
- `schema` contains JSON definitions of the different constituent formats of the spec (the _App Manifest_, the _Container Runtime Manifest_, and the `Fileset Manifest`). These JSON schemas also handle validation of the manifests through their Marshal/Unmarshal implementations.
|
||||
- `schema/types` contains various types used by the Manifest types to enforce validation
|
||||
- `ace` contains a tool intended to be run within an _Application Container Executor_ to validate that the ACE has set up the container environment correctly. This tool can be built into an ACI image ready for running on an executor by using the `build_aci` script.
|
||||
- `actool` contains a tool for building and validating images and manifests that meet the App Container specifications.
|
||||
|
||||
TODO(jonboulle): usage examples
|
||||
- app-container/ace/build_aci
|
||||
- bin/actool validate ...
|
||||
- bin/actool build ...
|
||||
## Building ACIs
|
||||
|
||||
`actool` can be used to build an Application Container Image from an application root filesystem (rootfs). It currently supports two modes: building an ACI from an existing [app manifest](SPEC.md#app-manifest), or building a [fileset image](SPEC.md#fileset-images) from a rootfs alone.
|
||||
|
||||
For example, to build a fileset containing certificate authorities, one could do the following:
|
||||
```
|
||||
$ actool build --fileset-name ca-certs /tmp/ca-certs/ ca-certs.aci
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
Since an ACI is simply an (optionally compressed) tar file, we can inspect the created file with simple tools:
|
||||
|
||||
```
|
||||
$ tar tvf ca-certs.aci
|
||||
drwxrwxr-x 1000/1000 0 2014-01-02 03:04 rootfs/
|
||||
drwxrwxr-x 1000/1000 0 2014-01-02 03:04 rootfs/certs/
|
||||
-rw-rw-r-- 1000/1000 3140 2014-01-02 03:04 rootfs/certs/ca-bundle.crt
|
||||
-rw-rw-r-- 1000/1000 3140 2014-01-02 03:04 rootfs/certs/ca-bundle.crt
|
||||
-rw-rw-r-- 1000/1000 1581 2014-01-02 03:04 rootfs/certs/example.com.crt
|
||||
-rw-r-xr-x root/root 174 2014-01-02 03:04 fileset
|
||||
$ tar xf ca-certs.aci fileset -O | python -m json.tool
|
||||
{
|
||||
"acKind": "FilesetManifest",
|
||||
"acVersion": "0.1.0",
|
||||
"arch": "amd64",
|
||||
"dependencies": null,
|
||||
"files": [
|
||||
"/certs/",
|
||||
"/ca-bundle.crt",
|
||||
"/example.com.crt",
|
||||
],
|
||||
"name": "ca-certs",
|
||||
"os": "linux"
|
||||
}
|
||||
```
|
||||
|
||||
To build an ACI image containing an application, supply a valid app manifest and the rootfs:
|
||||
|
||||
```
|
||||
$ actool build --app-manifest my-app.json my_app/rootfs my-app.aci
|
||||
```
|
||||
|
||||
Again, examining the ACI is simple, as is verifying that the app manifest was embedded appropriately:
|
||||
```
|
||||
$ tar tvf ca-certs.aci
|
||||
drwxrwxr-x 1000/1000 0 2014-01-02 03:04 rootfs/
|
||||
-rw-rw-r-- 1000/1000 1581 2014-01-02 03:04 rootfs/my_app
|
||||
-rw-r-xr-x root/root 174 2014-01-02 03:04 app
|
||||
```
|
||||
|
||||
```
|
||||
$ tar xf my-app.aci app -O | python -m json.tool
|
||||
{
|
||||
"acKind": "AppManifest",
|
||||
"acVersion": "1.0.0",
|
||||
"arch": "amd64",
|
||||
"exec": [
|
||||
"/my_app",
|
||||
],
|
||||
"group": "0",
|
||||
"name": "my_app",
|
||||
"os": "linux",
|
||||
"user": "0"
|
||||
}
|
||||
```
|
||||
|
||||
## Validating App Container implementations
|
||||
|
||||
`actool validate` can be used by implementations of the App Container Specification to check that files they produce conform to the expectations.
|
||||
|
||||
### Validating App Manifests, Fileset Manifests and Container Runtime Manifests
|
||||
|
||||
To validate one of the three manifest types in the specification, simply run `actool validate` against the file.
|
||||
|
||||
```
|
||||
$ actool ./app.json
|
||||
./app.json: valid AppManifest
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
Multiple arguments are supported, and the output can be silenced with `-quiet`:
|
||||
|
||||
```
|
||||
$ actool validate app1.json app2.json
|
||||
app1.json: valid AppManifest
|
||||
app2.json: valid AppManifest
|
||||
$ actool -quiet validate app2.json
|
||||
$ echo $?
|
||||
0
|
||||
```
|
||||
|
||||
`actool` will automatically determine which type of manifest it is checking (by using the `acKind` field common to all manifests), so there is no need to specify which type of manifest is being validated:
|
||||
```
|
||||
$ actool /tmp/my_fileset
|
||||
/tmp/my_fileset: valid FilesetManifest
|
||||
```
|
||||
|
||||
If a manifest fails validation, the first error encountered is returned, along with a non-zero exit status:
|
||||
```
|
||||
$ actool validate nover.json
|
||||
nover.json: invalid AppManifest: acVersion must be set
|
||||
$ echo $?
|
||||
1
|
||||
```
|
||||
|
||||
### Validating ACIs and layouts
|
||||
|
||||
Validating ACIs or layouts is very similar to validating manifests: simply run the `actool validate` subcommmand directly against an image or directory, and it will determine the type automatically:
|
||||
```
|
||||
$ actool validate app.aci
|
||||
app.aci: valid app container image
|
||||
$ actool validate app_layout/
|
||||
app_layout/: valid image layout
|
||||
```
|
||||
|
||||
To override the type detection and force `actool validate` to validate as a particular type (image, layout or manifest), use the `--type` flag:
|
||||
|
||||
```
|
||||
actool validate -type appimage hello.aci
|
||||
hello.aci: valid app container image
|
||||
```
|
||||
|
||||
### Validating App Container Executors (ACEs)
|
||||
|
||||
The (`ace`)[ace/] package contains a simple go application, the _ACE validator_, which can be used to validate app container executors by checking certain expectations about the environment in which it is run: for example, that the appropriate environment variables and mount points are set up as defined in the specification.
|
||||
|
||||
To use the ACE validator, first compile it into an ACI using the supplied `build_aci` script:
|
||||
```
|
||||
$ app-container/ace/build_aci
|
||||
|
||||
You need a passphrase to unlock the secret key for
|
||||
user: "Joe Bloggs (Example, Inc) <joe@example.com>"
|
||||
4096-bit RSA key, ID E14237FD, created 2014-03-31
|
||||
|
||||
Wrote main layout to bin/ace_main_layout
|
||||
Wrote unsigned main ACI bin/ace_validator_main.aci
|
||||
Wrote main layout hash bin/sha256-f7eb89d44f44d416f2872e43bc5a4c6c3e12c460e845753e0a7b28cdce0e89d2
|
||||
Wrote main ACI signature bin/ace_validator_main.sig
|
||||
|
||||
You need a passphrase to unlock the secret key for
|
||||
user: "Joe Bloggs (Example, Inc) <joe@example.com>"
|
||||
4096-bit RSA key, ID E14237FD, created 2014-03-31
|
||||
|
||||
Wrote sidekick layout to bin/ace_sidekick_layout
|
||||
Wrote unsigned sidekick ACI bin/ace_validator_sidekick.aci
|
||||
Wrote sidekick layout hash bin/sha256-13b5598069dbf245391cc12a71e0dbe8f8cdba672072135ebc97948baacf30b2
|
||||
Wrote sidekick ACI signature bin/ace_validator_sidekick.sig
|
||||
|
||||
```
|
||||
|
||||
As can be seen, the script generates two ACIs: `ace_validator_main.aci`, the main entrypoint to the validator, and `ace_validator_sidekick.aci`, a sidekick application. The sidekick is used to validate that an ACE implementation properly handles running multiple applications in a container (for example, that they share a mount namespace), and hence both ACIs should be run together in a layout to validate proper ACE behaviour. The script also generates detached signatures which can be verified by the ACE.
|
||||
|
||||
When running the ACE validator, output is minimal if tests pass, and errors are reported as they occur - for example:
|
||||
|
||||
```
|
||||
preStart OK
|
||||
main OK
|
||||
sidekick OK
|
||||
postStop OK
|
||||
```
|
||||
|
||||
or, on failure:
|
||||
```
|
||||
main FAIL
|
||||
==> file "/prestart" does not exist as expected
|
||||
==> unexpected environment variable "WINDOWID" set
|
||||
==> timed out waiting for /db/sidekick
|
||||
```
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
# App Container
|
||||
# App Container Specification
|
||||
|
||||
The "App Container" defines an image format, image discovery mechanism and execution environment that can exist in several independent implementations. The core goals include:
|
||||
|
||||
|
||||
@@ -61,8 +61,8 @@ func ValidateLayout(dir string) error {
|
||||
var amOK, fsmOK, rfsOK bool
|
||||
var am, fsm io.Reader
|
||||
walkLayout := func(fpath string, fi os.FileInfo, err error) error {
|
||||
fpath = strings.TrimPrefix(fpath, dir)
|
||||
name := filepath.Base(fpath)
|
||||
rpath := strings.TrimPrefix(fpath, dir)
|
||||
name := filepath.Base(rpath)
|
||||
switch name {
|
||||
case ".":
|
||||
case "app":
|
||||
@@ -83,7 +83,7 @@ func ValidateLayout(dir string) error {
|
||||
}
|
||||
rfsOK = true
|
||||
default:
|
||||
flist = append(flist, fpath)
|
||||
flist = append(flist, rpath)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -19,7 +19,7 @@ var (
|
||||
commands []*Command
|
||||
globalFlags = struct {
|
||||
Dir string
|
||||
Debug bool
|
||||
Quiet bool
|
||||
Help bool
|
||||
}{}
|
||||
transportFlags = struct {
|
||||
@@ -29,7 +29,7 @@ var (
|
||||
|
||||
func init() {
|
||||
globalFlagset.BoolVar(&globalFlags.Help, "help", false, "Print usage information and exit")
|
||||
globalFlagset.BoolVar(&globalFlags.Debug, "debug", false, "Print out more debug information to stderr")
|
||||
globalFlagset.BoolVar(&globalFlags.Quiet, "quiet", false, "Silence normal output")
|
||||
}
|
||||
|
||||
type Command struct {
|
||||
@@ -96,7 +96,9 @@ func getFlags(flagset *flag.FlagSet) (flags []*flag.Flag) {
|
||||
return
|
||||
}
|
||||
|
||||
func stderr(format string, a ...interface{}) {
|
||||
out := fmt.Sprintf(format, a...)
|
||||
fmt.Fprintln(os.Stderr, strings.TrimSuffix(out, "\n"))
|
||||
func stderr(quiet bool, format string, a ...interface{}) {
|
||||
if !quiet {
|
||||
out := fmt.Sprintf(format, a...)
|
||||
fmt.Fprintln(os.Stderr, strings.TrimSuffix(out, "\n"))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -37,6 +37,9 @@ func init() {
|
||||
|
||||
func buildWalker(root string, aw aci.ArchiveWriter, rootfs bool) filepath.WalkFunc {
|
||||
return func(path string, info os.FileInfo, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
relpath, err := filepath.Rel(root, path)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -85,22 +88,24 @@ func buildWalker(root string, aw aci.ArchiveWriter, rootfs bool) filepath.WalkFu
|
||||
}
|
||||
|
||||
func runBuild(args []string) (exit int) {
|
||||
q := globalFlags.Quiet
|
||||
if len(args) != 2 {
|
||||
stderr("build: Must provide directory and output file")
|
||||
stderr(q, "build: Must provide directory and output file")
|
||||
return 1
|
||||
}
|
||||
switch {
|
||||
case buildFilesetName != "" && buildAppManifest == "":
|
||||
case buildFilesetName == "" && buildAppManifest != "":
|
||||
default:
|
||||
stderr("build: must specify either --fileset-name or --app-manifest")
|
||||
stderr(q, "build: must specify either --fileset-name or --app-manifest")
|
||||
return 1
|
||||
}
|
||||
|
||||
root := args[0]
|
||||
tgt := args[1]
|
||||
ext := filepath.Ext(tgt)
|
||||
if ext != schema.ACIExtension {
|
||||
stderr("build: Extension must be %s (given %s)", schema.ACIExtension, ext)
|
||||
stderr(q, "build: Extension must be %s (given %s)", schema.ACIExtension, ext)
|
||||
return 1
|
||||
}
|
||||
|
||||
@@ -111,40 +116,51 @@ func runBuild(args []string) (exit int) {
|
||||
fh, err := os.OpenFile(tgt, mode, 0655)
|
||||
if err != nil {
|
||||
if os.IsExist(err) {
|
||||
stderr("build: Target file exists (try --overwrite)")
|
||||
stderr(q, "build: Target file exists (try --overwrite)")
|
||||
} else {
|
||||
stderr("build: Unable to open target %s: %v", tgt, err)
|
||||
stderr(q, "build: Unable to open target %s: %v", tgt, err)
|
||||
}
|
||||
return 1
|
||||
}
|
||||
defer func() {
|
||||
if exit != 0 && !buildOverwrite {
|
||||
fh.Close()
|
||||
os.Remove(tgt)
|
||||
}
|
||||
}()
|
||||
|
||||
tr := tar.NewWriter(fh)
|
||||
|
||||
var aw aci.ArchiveWriter
|
||||
if buildFilesetName != "" {
|
||||
aw, err = aci.NewFilesetWriter(buildFilesetName, tr)
|
||||
if err != nil {
|
||||
stderr("build: Unable to create FilesetWriter: %v", err)
|
||||
stderr(q, "build: Unable to create FilesetWriter: %v", err)
|
||||
return 1
|
||||
}
|
||||
} else {
|
||||
b, err := ioutil.ReadFile(buildAppManifest)
|
||||
if err != nil {
|
||||
stderr("build: Unable to read App Manifest: %v", err)
|
||||
stderr(q, "build: Unable to read App Manifest: %v", err)
|
||||
return 1
|
||||
}
|
||||
var am schema.AppManifest
|
||||
if err := am.UnmarshalJSON(b); err != nil {
|
||||
stderr("build: Unable to load App Manifest: %v", err)
|
||||
stderr(q, "build: Unable to load App Manifest: %v", err)
|
||||
return 1
|
||||
}
|
||||
aw = aci.NewAppWriter(am, tr)
|
||||
}
|
||||
|
||||
filepath.Walk(root, buildWalker(root, aw, buildRootfs))
|
||||
err = filepath.Walk(root, buildWalker(root, aw, buildRootfs))
|
||||
if err != nil {
|
||||
stderr(q, "build: Error walking rootfs: %v", err)
|
||||
return 1
|
||||
}
|
||||
|
||||
err = aw.Close()
|
||||
if err != nil {
|
||||
stderr("build: Unable to close Fileset image %s: %v", tgt, err)
|
||||
stderr(q, "build: Unable to close Fileset image %s: %v", tgt, err)
|
||||
return 1
|
||||
}
|
||||
|
||||
|
||||
@@ -27,17 +27,18 @@ func runDiscover(args []string) (exit int) {
|
||||
if len(args) < 1 {
|
||||
fmt.Fprintf(os.Stderr, "discover: at least one name required")
|
||||
}
|
||||
q := globalFlags.Quiet
|
||||
|
||||
for _, name := range args {
|
||||
labels, err := appFromString(name)
|
||||
if err != nil {
|
||||
stderr("%s: %s", name, err)
|
||||
stderr(q, "%s: %s", name, err)
|
||||
return 1
|
||||
}
|
||||
eps, err := discovery.DiscoverEndpoints(labels["name"], labels["ver"], labels["os"], labels["amd64"], transportFlags.Insecure)
|
||||
|
||||
if err != nil {
|
||||
stderr("error fetching %s: %s", name, err)
|
||||
stderr(q, "error fetching %s: %s", name, err)
|
||||
return 1
|
||||
}
|
||||
for _, list := range [][]string{eps.Sig, eps.ACI, eps.Keys} {
|
||||
|
||||
@@ -21,16 +21,17 @@ func runFetch(args []string) (exit int) {
|
||||
if len(args) < 1 {
|
||||
fmt.Fprintf(os.Stderr, "discover: at least one name required")
|
||||
}
|
||||
q := globalFlags.Quiet
|
||||
|
||||
for _, name := range args {
|
||||
labels, err := appFromString(name)
|
||||
if err != nil {
|
||||
stderr("%s: %s", name, err)
|
||||
stderr(q, "%s: %s", name, err)
|
||||
return 1
|
||||
}
|
||||
eps, err := discovery.DiscoverEndpoints(labels["name"], labels["ver"], labels["os"], labels["arch"], transportFlags.Insecure)
|
||||
if err != nil {
|
||||
stderr("error fetching %s: %s", name, err)
|
||||
stderr(q, "error fetching %s: %s", name, err)
|
||||
return 1
|
||||
}
|
||||
// TODO(philips): store the images..
|
||||
|
||||
@@ -43,8 +43,9 @@ func init() {
|
||||
}
|
||||
|
||||
func runValidate(args []string) (exit int) {
|
||||
q := globalFlags.Quiet
|
||||
if len(args) < 1 {
|
||||
stderr("must pass one or more files")
|
||||
stderr(q, "must pass one or more files")
|
||||
return 1
|
||||
}
|
||||
|
||||
@@ -52,7 +53,7 @@ func runValidate(args []string) (exit int) {
|
||||
vt := valType
|
||||
fi, err := os.Stat(path)
|
||||
if err != nil {
|
||||
stderr("unable to access %s: %v", path, err)
|
||||
stderr(q, "unable to access %s: %v", path, err)
|
||||
return 1
|
||||
}
|
||||
var fh *os.File
|
||||
@@ -62,7 +63,7 @@ func runValidate(args []string) (exit int) {
|
||||
case "":
|
||||
vt = typeImageLayout
|
||||
case typeManifest, typeAppImage:
|
||||
stderr("%s is a directory (wrong --type?)", path)
|
||||
stderr(q, "%s is a directory (wrong --type?)", path)
|
||||
return 1
|
||||
default:
|
||||
// should never happen
|
||||
@@ -71,7 +72,7 @@ func runValidate(args []string) (exit int) {
|
||||
} else {
|
||||
fh, err = os.Open(path)
|
||||
if err != nil {
|
||||
stderr("%s: unable to open: %v", path, err)
|
||||
stderr(q, "%s: unable to open: %v", path, err)
|
||||
return 1
|
||||
}
|
||||
}
|
||||
@@ -79,7 +80,7 @@ func runValidate(args []string) (exit int) {
|
||||
if vt == "" {
|
||||
vt, err = detectValType(fh)
|
||||
if err != nil {
|
||||
stderr("%s: error detecting file type: %v", path, err)
|
||||
stderr(q, "%s: error detecting file type: %v", path, err)
|
||||
return 1
|
||||
}
|
||||
}
|
||||
@@ -87,37 +88,36 @@ func runValidate(args []string) (exit int) {
|
||||
case typeImageLayout:
|
||||
err = aci.ValidateLayout(path)
|
||||
if err != nil {
|
||||
stderr("%s: invalid image layout: %v", path, err)
|
||||
} else if globalFlags.Debug {
|
||||
stderr("%s: valid image layout", path)
|
||||
stderr(q, "%s: invalid image layout: %v", path, err)
|
||||
exit = 1
|
||||
} else {
|
||||
stderr(q, "%s: valid image layout", path)
|
||||
}
|
||||
case typeAppImage:
|
||||
fr, err := maybeDecompress(fh)
|
||||
if err != nil {
|
||||
stderr("%s: error decompressing file: %v", path, err)
|
||||
stderr(q, "%s: error decompressing file: %v", path, err)
|
||||
return 1
|
||||
}
|
||||
tr := tar.NewReader(fr)
|
||||
err = aci.ValidateArchive(tr)
|
||||
fh.Close()
|
||||
if err != nil {
|
||||
stderr("%s: error validating: %v", path, err)
|
||||
return 1
|
||||
stderr(q, "%s: error validating: %v", path, err)
|
||||
exit = 1
|
||||
} else {
|
||||
stderr(q, "%s: valid app container image", path)
|
||||
}
|
||||
if globalFlags.Debug {
|
||||
stderr("%s: valid app container image", path)
|
||||
}
|
||||
continue
|
||||
case typeManifest:
|
||||
b, err := ioutil.ReadAll(fh)
|
||||
fh.Close()
|
||||
if err != nil {
|
||||
stderr("%s: unable to read file %s", path, err)
|
||||
stderr(q, "%s: unable to read file %s", path, err)
|
||||
return 1
|
||||
}
|
||||
k := schema.Kind{}
|
||||
if err := k.UnmarshalJSON(b); err != nil {
|
||||
stderr("error unmarshaling manifest: %v", err)
|
||||
stderr(q, "%s: error unmarshaling manifest: %v", path, err)
|
||||
return 1
|
||||
}
|
||||
switch k.ACKind {
|
||||
@@ -135,12 +135,13 @@ func runValidate(args []string) (exit int) {
|
||||
panic("bad ACKind")
|
||||
}
|
||||
if err != nil {
|
||||
stderr("%s: invalid %s: %v", path, k.ACKind, err)
|
||||
} else if globalFlags.Debug {
|
||||
stderr("%s: valid %s", path, k.ACKind)
|
||||
stderr(q, "%s: invalid %s: %v", path, k.ACKind, err)
|
||||
exit = 1
|
||||
} else {
|
||||
stderr(q, "%s: valid %s", path, k.ACKind)
|
||||
}
|
||||
default:
|
||||
stderr("%s: unable to detect filetype (try --type)", path)
|
||||
stderr(q, "%s: unable to detect filetype (try --type)", path)
|
||||
return 1
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
{
|
||||
"acVersion": "1.0.0",
|
||||
"acKind": "AppManifest",
|
||||
"name": "example.com/reduce-worker-1.0.0",
|
||||
"os": "linux",
|
||||
|
||||
@@ -65,12 +65,18 @@ func (am *AppManifest) assertValid() error {
|
||||
if am.ACKind != "AppManifest" {
|
||||
return types.ACKindError(`missing or bad ACKind (must be "AppManifest")`)
|
||||
}
|
||||
if am.ACVersion.Empty() {
|
||||
return errors.New(`acVersion must be set`)
|
||||
}
|
||||
if am.OS != "linux" {
|
||||
return errors.New(`missing or bad OS (must be "linux")`)
|
||||
}
|
||||
if am.Arch != "amd64" {
|
||||
return errors.New(`missing or bad Arch (must be "amd64")`)
|
||||
}
|
||||
if len(am.Exec) < 1 {
|
||||
return errors.New(`Exec cannot be empty`)
|
||||
}
|
||||
// TODO(jonboulle): assert hashes is not empty?
|
||||
return nil
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user