Pipeline: series of named steps.
Step: named set of operations (make, script, something extensible). Optionally names an artifact that is the result of the step.
Artifact: resulting output from a step. strawboss may skip step execution if it already has an artifact cached from that step.
Configuration is two-part. Pipeline, step, and artifact definition are part of the project configuration (.strawboss.json? yaml?). Environment configuration lives on the strawboss server (supplies DB info, etc.).
REST API?
Step execution happens within the root directory of a fresh copy of the repo. Commit identifiers (hash/ref/etc.) are supplied when a build is triggered and the fresh copy is checked out at that reference.

Projects

StrawBoss must know about the projects it builds. For each project, StrawBoss must be configured with a unique project name and a URL or repo-specification that can be passed to git clone to clone the project.

For more details see the Service Project Definition section.

Artifacts Repo

The artifacts repository is a directory on the filesystem used to store build artifacts and information about build status. The top-level artifacts repo directory contains project build directories: one directory for every project that StrawBoss has built. Each project directory contains multiple sub-directories. Each

Configuration

StrawBoss configuration file

StrawBoss expects to find strawboss.config.json in the working directory of the strawboss executable. This is the configuration file for StrawBoss itself. The contents are expected to be a valid JSON object. The top level keys are:

artifactsRepo: A string denoting the path to the artifacts repository directory.
users: the array of user definition objects. Each user object is required to have username and hashedPwd keys, both string.
tokens: an array of string, each representing a valid auth token that has been issued to a client.
projects: an array of project definitions (detailed below).

All are required.

Service Project Definition

Project definitions are JSON objects with the following keys:

name (required): A unique name for the project. Names are case sensitive. There are no hard limits on names but the recommended convention is alpha-numeric using dashes or underscores for word separation (e.g. foo-confabulator or web_refractulator, etc.).
repo (required): A valid git URL or repo-specification that can be passed to git clone to clone the project.
defaultBranch (optional): If a run request is submitted without specifying a branch or commit reference, use this branch (defaults to master).
cfgFilePath (optional): path to the StrawBoss project configuration file (see the Project Configuration section). If this is a relative path it is resolved relative to the project root directory. (defaults to strawboss.json).
envVars (optional): an object whose key-value pairs are expected to be strings. Each pair will be added as an environment variable to the runtime environment of builds for this project. (defaults to {}).

Project Configuration

The build configuration for each project lives in the project itself. In the root directory of the project repo StrawBoss expects to find a project configuration file named strawboss.json (this location can be configured, see the Service Project Definition section). The project configuration file tells StrawBoss what steps of a project are available to be built, and how to initiate the build for each step. Steps can be dependant on each other, as long as they do not define any circular references.

The contents of a project configurations files is expected to be a valid JSON object. The top level keys are:

name (required): the unique name for the project. This should be the same name as configured in the StrawBoss service configuration file.
steps (required): an object of step definitions. Each of this object's keys is the name of a step. The corresponding value is the step definition. (see the Step Definition section for details).
versionCmd (optional): a command to be run in a shell (sh-compatible) that is expected to print the current version of the project on stdout. (defaults to git describe --tags --always).

Build Process

When performing a build, StrawBoss:

creates a temporary workspace for this build
clones the repo into the workspace
checkout the revision or branch requested for this run
load the project's StrawBoss configuration file.
merge environment variables defined in the project configuration
run versionCmd to get the current project version. The result is stored in the VERSION environment variable.
check the environment variables against expectedEnv
check that all the steps named in depends have already been run and run them if they have not. For each step named in depends an environment variable is added named <step-name>_DIR that contains the absolute path to the artifacts repo for that step at this version. This is intended to be used to reference artifacts from other steps, e.g. ${build_DIR}/site-contents.zip.
stepCmd is executed in workingDir. Environment variables in cmdInput are resolved and the resulting string are fed line-by-line into the process as stdin.
the files named in artifacts are copied into the artifacts repo for this step and version.

Step Definition

Step definitions are JSON objects with the following keys:

workingDir (optional): the working directory (relative to the project root folder) for all commands executed as part of this step. (defaults to '.', the project root directory).
stepCmd (optional): the command to execute for this step. (defaults to sh)
cmdInput (optional): an array of string that will be concatenated with newlines separating each string and piped as input to the command for this step. It is possible to include newlines in the strings, but not required. Environment variables included in this input are resolved before it is passed to stdin as stepCmd does not have to be a shell and is not expected to be able to do this interpolation itself. (defaults to []).
artifacts (optional): an array of strings representing artifacts that will be generated by this step and should be saved for future use. Each string should represent a path to a file (relative to the project root directory). StrawBoss will gather all of these files and store them in the artifacts repo for this step and build. These paths may contain environment variable references. (defaults to []).
depends (optional): an array of strings representing step names upon which this step depends. (defaults to []).
expectedEnv (optional): an array of strings representing environment variable keys that should be present in the environment when executing this step. If there is an environment variable that is listed here but not present in the build environment it will cause StrawBoss to fail this step. (defaults to []).
dontSkip (optional): boolean. StrawBoss remembers the steps it has run and caches the artifacts generated by them. Future builds for the same version that request this step normally will not cause StrawBoss to re-run the step. If dontSkip is set to true, the output of this step will always be run when it is referenced, regardless of previous cached results.

Architecture

The following describes the internal architecture of StrawBoss. This section is written primarily for those wanting to work on StrawBoss itself and is not nessecary for the setup and use of StrawBoss.

Artifacts repo

In the top-level of the artifacts repo are project directories.

Project Directories

Project directories contain step directories and cached project configuration files.

Cached configuration files.

The cached project configuration files follow this naming convention: configuration.<version>.json. StrawBoss uses the file modification time to determine which configuration file is the most recent. These cached versions of the project configration are only intended to be used in cases where StrawBoss is not building anything and doesn't check out a copy of the repo. For example, when a client queries the REST API for the list of steps in a project, StrawBoss will consult the most recently modified cached copy of the project configuration rather than cloning the entire repo just to answer this question. Whenever StrawBoss has a copy of the repo, it should look for the actual configuration file in that version of the repo instead of consulting the cached configuration files.

Step and Version Directories

A step directory contains a list of version directories using the version string as the directory names and a list of status JSON files also named after the version (<version>.status.json).

A version directory contains the artifacts from this step of the build, a stdout.log file, and a stderr.log file containing the outputs of the build process.

Worker Processes

When in server mode StrawBoss has a multi-process model. There is a supervisory process that serves the REST API, receives build requests, and spawns worker processes. The worker processes are responsible for actually running the builds. The supervisory process registers a handler for SIGCHLD, using the handler to update the supervisor's knowledge of the build results and wait for the process to be cleaned up.

When launched in single-build mode there is no supervisory process. The main process directly executes the requested build steps.