* Addressing breaking changes in migration from Nim 0.18 to 0.19.
* Finishing the initial pass at the refactor required to include
docker-based builds.
* Regaining confidence in the existing functionality by getting all
tests passing again after docker introduction (still need new tests to
cover new docker functionality).
* Added endopint to return the logs from a run.
* Refactored the `pathVar & "/" & pathvar` pattern into `pathVar / pathVar`
using the `ospaths` module. Cleaner code, more resistant to extra `/` bugs.
* Added endpoints and core methods to list artifacts for a build, as well as to
retrieve specific artifacts.
* Fixed a problem with the `complete` status being overloaded. The problem was
that in the case of a multi-step build all of the prerequisite steps will
return a state of `complete` which will get recorded in the status file for
the run. The run will continue, but anyone watching the run state file (via
the API for example) had no definitive way to tell the difference between a
sub-step completing and the requested (last) step completing. This was caught
in the functional tests (race condition based on when it polls for the run
status). The fix was to introduce a new build state: `stepComplete`. The
inner `doRun` procedure uses this instead of `complete`. Now the only place
that `complete` is set is at the end of the original call to `run`, right
before the worker terminates. It checks the last result (from the originally
requested step) and if this result is `stepComplete` it "finishes" the build
by setting the state to `complete`. Because this is the only place where
`complete` is set, an observer is now guaranteed not to see `complete` until
all steps have run successfully.
* Fixed a long-standing bug with the request handling logic in error cases
(like requested resources not being available). Issue has something to do
with the way that `except` blocks become special when in an async context.
The main jester routes block invokes the handlers in an async context. The
effect is that one `except` block is fine, but adding more than one (to catch
different exception types, for example) causes the return type of the route
handler to change and not match what the outer block is expecting (a Future).
The fix here is to wrap any exception discrimination within a single outer
except block, re-raise the exception, and catch it inside this new
synchronous context. Ex:
```nim
try: someCall(mayFail)
except:
try: raise getCurrentException()
except ExceptionType1:
# do whatever...
except ExceptionType2:
# do whatever
except:
# general catch-all
return true
```
The return at the end is also part of the story. Jester's match handler
allows a route to defer making a decision about whether it matches. If you
return true from a route block Jester accepts the result as a matched route.
If you return false, Jester discards the result and looks for another
matching route. Normally this is taken care of by the `resp` templates
provided by Jester, but invoking those templates within the except blocks
also causes problems, so we manually setup the response and `return true` to
tell Jester that, yes this route matched, use this response.
* Moved the `/service/debug/ping` endpoint back to `/ping` and removed the
debug-only fence. I envision this as being useful as a simple healthcheck URL.
There is some bug building in the docker image we use to build the project with
the latest version of https://github.com/yglukhov/nim-jwt so I'm pinning it to
commit hash 549aa1eb13b8ddc0c6861d15cc2cc5b52bcbef01 for now. Later versions
add an ifdef branch to support libssl 1.1 but for some reason that ifdef is set
wrong and it tries to build against the 1.1 API even though the image only has
the 1.0 API. I'm crossing my fingers and hoping that our base image supports
libssl 1.1 before I need to update this library.
* Fixed the formatting of command line logging of strawboss workers.
* Fixed a bug in the (de)serialization of log levels in the strawboss service
config file.
* Pulled `parseBuildStatus` logic out of `loadBuildStatus` so that we could
parse a JSON that didn't come from a file.
* Added `parseRun` for Run objects.
* Moved `/ping` to `/service/debug/ping` for symmetry with
`/service/debug/stop`
* Added functional tests of full builds.
* Output from the main strawboss executable is properly directed to stdout
and stderr.
* Added threshold logging to strawboss core functions.
* Fixed a bug in the way dependent steps were detected and executed.
The logic for checking if prior steps had already been executed was only
executed once when the initial step was prepared, not for any of the
dependent steps. This logic has been moved into the main work block for
executing steps.
* Renamed `initiateRun` to `run` and `runStep` to `doRun` to be more accurate.
* Dependent steps get their owng, independent copy of the workspace.
* Updated the test project to provide a test target.
* Implemented periodic maintenance window.
* Moved worker creation into the core module.
* Worker processes no longer create run requests, but read queued requests from
the file system.
* Build status and logs have been moved into the StrawBoss data directory.
* An initial build status is recorded when the job is queued.
* Build status is recorded for build references as well as actual versions.
So there will be a build status for "master", for example, that is
overwritten whenever "master" is built for that step.
* RunRequests now include a timestamp.
* Added a Run object to contain both a RunRequest and the corresponding
BuildStatus for that run.
* API endpoints that talk about runs now return Run objects instead of
RunRequests.
* Moved all data layer operations into the core module so that the
"database API" only lives in one place.
* Rename artifactsRepo -> buildDataDir to be more explicit about the fact that
it holds more than just the artifacts.
* Revert removal of run ids.
* Move Worker definition into core as part of making the core responsible for
accepting run requests.
* Make the core module more responsible for internal details of data structure
and storage. External callers should not need to construct paths to
artifacts, versions, etc. but should be able to call method in the core
module to do this work for them.
* The working directory no longer contains anything but the checked-out code.
All StrawBoss-specific data is stored by StrawBoss elsewhere.
* Add a regular maintenance cycle to the server module.
StarBoss is meant for building things checked into the repo It is also designed
around repeatable builds. So it makes the assumption that running a build step
for a specific version of a project will always result in the same output. So
runs are identified by the project, build step, and version.
We were expecting to find the path to the `strawboss` binary implicitly from
the environment, which meant that configuration was also implicit, and required
more setup. Now the path to the binary is explicit in the StrawBoss runtime
configuration, and the path to the configuration file can also be explicitly given.
* Split the `test` nimble task into `unittest` and `functest`, with
corresponding test directories and test runners.
* Added documentation in README regarding building and testing StrawBoss.
* Created a small, simple test project for use in the functional tests.
* Added a `keepEnv` template in the server unit test code to make it easy to
preserve the working environment for a single unit test to invistigate
failures manually.
