Add show-dupes command, fix BareExcept warnings.

* Add `state` on `Issue` to be able to query the state of an issue even
  if you only have a reference to this issue and don't have a reference
  to the context or issues table. This does not change the persisted
  format of the issue. On disk the state of an issue is still
  represented by it's location in the file hierarchy.

* Refactored libpit to use zero_functional instead of sequtils.

README.md

@@ -0,0 +1,138 @@
+# Personal Issue Tracker
+
+This is [Jonathan Bernard's](mailto:jonathan@jdbernard.com) personal issue
+tracker. In it's current form it is essentially a way to keep an curated list of
+TODO's, organizing them by workflow category (todo, todo-today, dormant, etc.)
+and context (Personal, Work, etc.).
+
+## Categories
+
+`pit` organizes issues into the following workflow categories:
+
+- `current` - actively in progress
+- `todo` - to be addressed in the future
+- `todo-today` - chosen to be addressed today
+- `pending` - blocked by some third party
+- `dormant` - long-term things I don't want to forget but don't need in front
+  of me every day.
+- `done`
+
+In my typical workflow the `todo` category serves as a collection point for
+things I want to keep track of. Then on a a daily basis I review issues in the
+`todo` category and move a selection to the `todo-today` category. I also try
+to keep the total number of issues in the `todo` below about a dozen. If there
+are more than a dozen things in my `todo` category I will identify the lowest
+priority items and move them to the `dormant` category.
+
+## Issue Properties
+
+`pit` allows arbitrary properties to be attached to issues in the form of
+key-value pairs. On the command line these can be provided via the `-p` or
+`--properties` parameter in the form
+`-p <prop1Name>:<prop1Value>;<prop2Name>:<prop2Value>[;...]`
+
+There are a couple of properties that pit will recognize automatically:
+
+- `context`: the context organization feature is implemented using issue
+  properties.
+- `created`: `pit` uses this property to timestamp an issue when it is created.
+- `completed`: `pit` uses this property to timestamp an issue when it is moved
+  to the `done` category.
+- `pending`: `pit` looks to this property to provide extra information about
+  issues in the `pending` category. Typically I use this to note who or what is
+  blocking the issue and why.
+
+Some other common properties I use are:
+
+- `resolution`: for short notes about why an issue was moved to `done`,
+  especially if it the action wasn't taken or if it is not completely clear
+  that this issue was completed.
+
+## Configuration Options
+
+`pit` allows configuration via command-line options and via a configuration
+file. There is some overlap between the two methods of configuring `pit`, but
+it is not a complete mapping.
+
+### Config File
+
+`pit` looks for a JSON configuration file in the following places (in order):
+
+1. From a file path passed on the command line via the `--config <cfgFile>` parameter,
+2. `./.pitrc`, in the current working directory,
+3. From a file path set in the `PITRC` environment variable.
+4. `$HOME/.pitrc`, in the user's home directory.
+
+
+#### Sample Config File
+
+This example illustrates all of the possible configuration options.
+
+```json
+{
+  "api": {
+    "apiKeys": [
+      "50cdcb660554e2d50fd88bd40b6579717bf00643f6ff57f108baf16c8c083f77",
+      "e4fc1aac49fc1f2f7f4ca6b1f04d41a4ccdd58e13bb53b41da97703d47267ceb",
+    ]
+  },
+  "cli": {
+    "defaultContext": "personal",
+    "verbose": false,
+    "termWidth": 120,
+    "triggerPtk": true
+  },
+  "contexts": {
+    "nla-music": "New Life Music",
+    "nla-youth-band": "New Life Youth Band",
+    "acn": "Accenture",
+    "hff": "Hope Family Fellowship"
+  },
+  "tasksDir": "/mnt/c/Users/Jonathan Bernard/synced/tasks"
+}
+```
+
+#### Explanation of configurable options.
+
+In general, options supplied on the CLI directly will override options supplied
+in the configuration file. All options are optional unless stated otherwise.
+
+* `api`: configuration options specific to the API service.
+
+  - `apiKeys`: a list of Bearer tokens accepted by the API for the purpose of
+    authenticating API requests.
+
+* `cli`: configuration options specific to the CLI.
+
+  - `defaultContext`: if present all invokations to the CLI will
+    be in this context. This is like adding a `--context <defaultContext>`
+    parameter to every CLI invocation. Any actual `--context` parameter will
+    override this value.
+
+  - `verbose`: Show issue details when listing issues (same as
+    `--verbose` flag).
+
+  - `termWidth`: Set the expected width of the terminal (for wrapping text).
+
+  - `triggerPtk`: If set to `true`, invoke the `ptk` command to start and stop
+    timers when issues move to the `current` and `done` categories
+    respectively.
+
+* `contexts`: `pit` allows issues to be organized into different contexts via
+  a `context` property on the issue. The CLI groups issues according to
+  context. When printing contexts the CLI will take the value from the issues'
+  `context` properties and capatalize it. In some cases you may wish to have a
+  different display value for a context. I like to use abbreviations for long
+  context names to reduce the need to type, `hff` for "Hope Family Fellowship",
+  for example. The `contexts` config option allows you to provide a map of
+  context values to context display names See the sample file below for an
+  example.
+
+  Note that this mapping does not have to have entries for all contexts, only
+  those you wish to provide with an alternate display form. For example, in the
+  configuration sample above the default context is `personal`, a value not
+  present in the `contexts` configuration. `personal` will be displayed as
+  "Personal"; it does not need an alternate display name.
+
+* `tasksDir` **required**: a file path to the root directory for the issue
+  repository (same as `--tasks-dir` CLI parameter).

nimble.lock

@@ -0,0 +1,174 @@
+{
+  "version": 2,
+  "packages": {
+    "asynctools": {
+      "version": "0.1.1",
+      "vcsRevision": "0e6bdc3ed5bae8c7cc9e03cfbf66b7c882a908a7",
+      "url": "https://github.com/cheatfate/asynctools",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "54314dceabb06b20908ecb0f2c007e9ff3aaa054"
+      }
+    },
+    "isaac": {
+      "version": "0.1.3",
+      "vcsRevision": "45a5cbbd54ff59ba3ed94242620c818b9aad1b5b",
+      "url": "https://github.com/pragmagic/isaac/",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "05c3583a954715d84b0bf1be97f9a503249e9cdf"
+      }
+    },
+    "uuids": {
+      "version": "0.1.11",
+      "vcsRevision": "8cb8720b567c6bcb261bd1c0f7491bdb5209ad06",
+      "url": "https://github.com/pragmagic/uuids/",
+      "downloadMethod": "git",
+      "dependencies": [
+        "isaac"
+      ],
+      "checksums": {
+        "sha1": "393f5fcefbc8ad3cf167e59760144208ff8f9f76"
+      }
+    },
+    "unicodedb": {
+      "version": "0.11.2",
+      "vcsRevision": "c70f8bc8c7373265670e0575bc5eda36fe3761b0",
+      "url": "https://github.com/nitely/nim-unicodedb",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "612c5955f91bd90a263ce914d1d5de74a33af3c6"
+      }
+    },
+    "regex": {
+      "version": "0.20.1",
+      "vcsRevision": "66f144f935cc73977c61185fab15a3147bf117ff",
+      "url": "https://github.com/nitely/nim-regex",
+      "downloadMethod": "git",
+      "dependencies": [
+        "unicodedb"
+      ],
+      "checksums": {
+        "sha1": "ea9b6600443e73b1ea89a477c7a5d1fce742c9da"
+      }
+    },
+    "docopt": {
+      "version": "0.7.0",
+      "vcsRevision": "17803d1205f9e752cce03a66b0a29b710520398e",
+      "url": "https://github.com/docopt/docopt.nim",
+      "downloadMethod": "git",
+      "dependencies": [
+        "regex"
+      ],
+      "checksums": {
+        "sha1": "21150284640b882fa91147181c52da3e5bb44df8"
+      }
+    },
+    "filetype": {
+      "version": "0.9.0",
+      "vcsRevision": "1fe1e7d988cd802abc26505efb5a91891bd6f53e",
+      "url": "https://github.com/jiro4989/filetype",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "d2242b94eeb0f6d3810a8c71af4664f28853e1be"
+      }
+    },
+    "zero_functional": {
+      "version": "1.3.0",
+      "vcsRevision": "edf3b7f59119f75706da435c2b7f080a0cf4960c",
+      "url": "https://github.com/zero-functional/zero-functional",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "2dc01ca0925ac1c2dcb46a0c6d9c93c57a9cddec"
+      }
+    },
+    "update_nim_package_version": {
+      "version": "0.2.0",
+      "vcsRevision": "5a78579fd7f88014263aed38c60327c85f6f8bcf",
+      "url": "https://git.jdb-software.com/jdb/update-nim-package-version",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "37052b7ce30d5493ef24253a82a6087350b4eabb"
+      }
+    },
+    "data_uri": {
+      "version": "1.0.2",
+      "vcsRevision": "f43ac66e44c37edd3cc7282d75d6fa2fa031b2ec",
+      "url": "",
+      "downloadMethod": "git",
+      "dependencies": [
+        "docopt",
+        "filetype",
+        "zero_functional",
+        "update_nim_package_version"
+      ],
+      "checksums": {
+        "sha1": "949c11ffab4e85ff538b0bd3e5bb193f118b56d7"
+      }
+    },
+    "timeutils": {
+      "version": "0.5.4",
+      "vcsRevision": "a9308cbaf3c89496b5832ddd18404dc0debe66a2",
+      "url": "https://git.jdb-software.com/jdb/nim-time-utils.git",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "9ecd1020f5644bc59acb2f44ca9d30f4c7f066d3"
+      }
+    },
+    "cliutils": {
+      "version": "0.8.0",
+      "vcsRevision": "b1cc4fbe51d5e617789363efe716793ebe5bc5f1",
+      "url": "https://git.jdb-software.com/jdb/nim-cli-utils",
+      "downloadMethod": "git",
+      "dependencies": [
+        "docopt"
+      ],
+      "checksums": {
+        "sha1": "5b114094c314007fa6f15e62852d62a58a3cbb62"
+      }
+    },
+    "langutils": {
+      "version": "0.4.0",
+      "vcsRevision": "8122660da3fc78132b823e76c9e990fd92802b0e",
+      "url": "https://git.jdb-software.com/jdb/nim-lang-utils.git",
+      "downloadMethod": "git",
+      "dependencies": [],
+      "checksums": {
+        "sha1": "2da09deb0e0bfc186000f0d941d06dd974bf6e58"
+      }
+    },
+    "httpbeast": {
+      "version": "0.4.1",
+      "vcsRevision": "abc13d11c210b614960fe8760e581d44cfb2e3e9",
+      "url": "https://github.com/dom96/httpbeast",
+      "downloadMethod": "git",
+      "dependencies": [
+        "asynctools"
+      ],
+      "checksums": {
+        "sha1": "b23e57a401057dcb9b7fae1fb8279a6a2ce1d0b8"
+      }
+    },
+    "jester": {
+      "version": "0.5.0",
+      "vcsRevision": "a21b36a02b7745d6cdcda32d4ab3fba328cda17a",
+      "url": "https://github.com/dom96/jester/",
+      "downloadMethod": "git",
+      "dependencies": [
+        "httpbeast",
+        "asynctools"
+      ],
+      "checksums": {
+        "sha1": "a192ca25bfc05d5de5c9a5fafca3b0cee47d82d2"
+      }
+    }
+  },
+  "tasks": {}
+}

pit.nimble

@@ -1,15 +1,31 @@
 # Package
 
-include "src/pitpkg/version.nim"
-
-version       = PIT_VERSION
+version       = "4.23.4"
 author        = "Jonathan Bernard"
 description   = "Personal issue tracker."
 license       = "MIT"
 srcDir        = "src"
+installExt    = @["nim"]
 bin           = @["pit", "pit_api"]
 
 # Dependencies
 
-requires @[ "nim >= 0.18.0", "cliutils 0.4.1", "docopt 0.6.5", "jester 0.2.0",
-  "timeutils 0.3.0", "uuids 0.1.9" ]
+requires @[
+  "nim >= 1.4.0",
+  "docopt >= 0.6.8",
+  "jester >= 0.5.0",
+  "uuids >= 0.1.10",
+  "zero_functional"
+]
+
+# Dependencies from git.jdb-software.com/nim-jdb/packages
+requires @[
+  "cliutils >= 0.8.1",
+  "langutils >= 0.4.0",
+  "timeutils >= 0.5.4",
+  "data_uri > 1.0.0",
+  "https://git.jdb-software.com/jdb/update-nim-package-version >= 0.2.0"
+]
+
+task updateVersion, "Update the version of this package.":
+  exec "update_nim_package_version pit 'src/pitpkg/cliconstants.nim'"

src/pit.nim

@@ -1,25 +1,30 @@
 ## Personal Issue Tracker CLI interface
 ## ====================================
 
-import cliutils, docopt, json, logging, options, os, ospaths, sequtils,
-  tables, terminal, times, timeutils, unicode, uuids
+import std/[algorithm, logging, options, os, sequtils, wordwrap, tables,
+            terminal, times, unicode]
+import cliutils, data_uri, docopt, json, timeutils, uuids
 
-import strutils except capitalize, toUpper, toLower
+from nre import re
+import strutils except alignLeft, capitalize, strip, toUpper, toLower
 import pitpkg/private/libpit
+import pitpkg/cliconstants
 export libpit
 
-include "pitpkg/version.nim"
-
 type
   CliContext = ref object
     cfg*: PitConfig
     contexts*: TableRef[string, string]
     defaultContext*: Option[string]
-    tasksDir*: string
     issues*: TableRef[IssueState, seq[Issue]]
     termWidth*: int
     triggerPtk*, verbose*: bool
 
+let EDITOR =
+  if existsEnv("EDITOR"): getEnv("EDITOR")
+  else: "vi"
+
+
 proc initContext(args: Table[string, Value]): CliContext =
   let pitCfg = loadConfig(args)
 
@@ -30,23 +35,23 @@ proc initContext(args: Table[string, Value]): CliContext =
   let cliCfg = CombinedConfig(docopt: args, json: cliJson)
 
   result = CliContext(
+    cfg: pitCfg,
     contexts: pitCfg.contexts,
     defaultContext:
       if not cliJson.hasKey("defaultContext"): none(string)
       else: some(cliJson["defaultContext"].getStr()),
     verbose: parseBool(cliCfg.getVal("verbose", "false")) and not args["--quiet"],
     issues: newTable[IssueState, seq[Issue]](),
-    tasksDir: pitCfg.tasksDir,
-    termWidth: parseInt(cliCfg.getVal("term-width", "80")),
+    termWidth: parseInt(cliCfg.getVal("termWidth", "80")),
     triggerPtk: cliJson.getOrDefault("triggerPtk").getBool(false))
 
 proc getIssueContextDisplayName(ctx: CliContext, context: string): string =
   if not ctx.contexts.hasKey(context):
-    if context.isNilOrWhitespace: return "<default>"
+    if context.isEmptyOrWhitespace: return "<default>"
     else: return context.capitalize()
   return ctx.contexts[context]
 
-proc formatIssue(ctx: CliContext, issue: Issue): string =
+proc formatIssue*(issue: Issue): string =
   result = ($issue.id).withColor(fgBlack, true) & "\n"&
            issue.summary.withColor(fgWhite) & "\n"
 
@@ -60,56 +65,85 @@ proc formatIssue(ctx: CliContext, issue: Issue): string =
 
 
   result &= "--------".withColor(fgBlack, true) & "\n"
-  if not issue.details.isNilOrWhitespace:
+  if not issue.details.isEmptyOrWhitespace:
     result &= issue.details.strip.withColor(fgCyan) & "\n"
 
-proc formatSectionIssue(ctx: CliContext, issue: Issue, width: int, indent = "",
-                verbose = false): string =
-
-  result = ""
-
-  var showDetails = not issue.details.isNilOrWhitespace and verbose
-
-  # Wrap and write the summary.
-  var wrappedSummary = (" ".repeat(5) & issue.summary).wordWrap(width - 2).indent(2 + indent.len)
-  wrappedSummary = wrappedSummary[(6 + indent.len)..^1]
-
-  result = (indent & ($issue.id)[0..<6]).withColor(fgBlack, true)
-  result &= wrappedSummary.withColor(fgWhite)
-
-  if issue.tags.len > 0:
-    let tagsStr = "(" & issue.tags.join(", ") & ")"
-    if (result.splitLines[^1].len + tagsStr.len + 1) > (width - 2):
-      result &= "\n" & indent
-    result &= " " & tagsStr.withColor(fgGreen)
-
-
-  if issue.hasProp("pending"):
-    let startIdx = "Pending: ".len
-    var pendingText = issue["pending"].wordWrap(width - startIdx - 2)
-                                      .indent(startIdx)
-    pendingText = ("Pending: " & pendingText[startIdx..^1]).indent(indent.len + 2)
-    result &= "\n" & pendingText.withColor(fgCyan)
-
-  if showDetails:
-    result &= "\n" & issue.details.strip.indent(indent.len + 2).withColor(fgCyan)
-
   result &= termReset
 
-proc formatSectionIssueList(ctx: CliContext, issues: seq[Issue], width: int,
-        indent: string, verbose: bool): string =
+proc formatSectionIssue*(
+    issue: Issue,
+    width: int = 80,
+    indent = "",
+    verbose = false): string =
+
+  result = (indent & ($issue.id)[0..<6]).withColor(fgBlack, true) & " "
+
+  let showDetails = not issue.details.isEmptyOrWhitespace and verbose
+
+  let summaryIndentLen = indent.len + 7
+  let summaryWidth = width - summaryIndentLen
+
+  let summaryLines = issue.summary
+    .wrapWords(summaryWidth)
+    .splitLines
+
+  result &= summaryLines[0].withColor(fgWhite)
+
+  for line in summaryLines[1..^1]:
+    result &= "\p" & line.indent(summaryIndentLen)
+
+  var lastLineLen = summaryLines[^1].len
+
+  if issue.hasProp("delegated-to"):
+    if lastLineLen + issue["delegated-to"].len + 1 < summaryWidth:
+      result &= " " & issue["delegated-to"].withColor(fgMagenta)
+      lastLineLen += issue["delegated-to"].len + 1
+    else:
+      result &= "\p" & issue["delegated-to"]
+        .withColor(fgMagenta)
+        .indent(summaryIndentLen)
+      lastLineLen = issue["delegated-to"].len
+
+  if issue.tags.len > 0:
+    let tagsStrLines = ("(" & issue.tags.join(", ") & ")")
+      .wrapWords(summaryWidth)
+      .splitLines
+
+    if tagsStrLines.len == 1 and
+       (lastLineLen + tagsStrLines[0].len + 1) < summaryWidth:
+      result &= " " & tagsStrLines[0].withColor(fgGreen)
+      lastLineLen += tagsStrLines[0].len + 1
+    else:
+      result &= "\p" & tagsStrLines
+        .mapIt(it.indent(summaryIndentLen))
+        .join("\p")
+        .withColor(fgGreen)
+      lastLineLen = tagsStrLines[^1].len
+
+  if issue.hasProp("pending"):
+    result &= "\p" & ("Pending: " & issue["pending"])
+      .wrapwords(summaryWidth)
+      .withColor(fgCyan)
+      .indent(summaryIndentLen)
+
+  if showDetails:
+    result &= "\p" & issue.details
+      .strip
+      .withColor(fgBlack, bright = true)
+      .indent(summaryIndentLen)
+
+  result &= termReset
+
+proc formatSectionIssueList*(
+    issues: seq[Issue],
+    width: int = 80,
+    indent: string = "",
+    verbose: bool = false): string =
 
   result = ""
-  var topPadded = true
   for i in issues:
-    var issueText = ctx.formatSectionIssue(i, width, indent, verbose)
-    if issueText.splitLines.len > 1:
-      if topPadded: result &= issueText & "\n\n"
-      else: result &= "\n" & issueText & "\n\n"
-      topPadded = true
-    else:
-      result &= issueText & "\n"
-      topPadded = false
+    var issueText = formatSectionIssue(i, width, indent, verbose)
+    result &= issueText & "\n"
 
 proc formatSection(ctx: CliContext, issues: seq[Issue], state: IssueState,
                    indent = "", verbose = false): string =
@@ -129,17 +163,20 @@ proc formatSection(ctx: CliContext, issues: seq[Issue], state: IssueState,
         indent & ctx.getIssueContextDisplayName(context) & ":" &
         termReset & "\n\n"
 
-      result &= ctx.formatSectionIssueList(ctxIssues, innerWidth - 2, indent & "  ", verbose)
+      result &= formatSectionIssueList(ctxIssues, innerWidth - 2, indent & "  ", verbose)
       result &= "\n"
 
-  else: result &= ctx.formatSectionIssueList(issues, innerWidth, indent, verbose)
+  else: result &= formatSectionIssueList(issues, innerWidth, indent, verbose)
 
 proc loadIssues(ctx: CliContext, state: IssueState) =
-  ctx.issues[state] = loadIssues(ctx.tasksDir / $state)
+  ctx.issues[state] = loadIssues(ctx.cfg.tasksDir, state)
+
+proc loadOpenIssues(ctx: CliContext) =
+  ctx.issues = newTable[IssueState, seq[Issue]]()
+  for state in [Current, TodoToday, Todo, Pending, Todo]: ctx.loadIssues(state)
 
 proc loadAllIssues(ctx: CliContext) =
-  ctx.issues = newTable[IssueState, seq[Issue]]()
-  for state in IssueState: ctx.loadIssues(state)
+  ctx.issues = ctx.cfg.tasksDir.loadAllIssues()
 
 proc filterIssues(ctx: CliContext, filter: IssueFilter) =
   for state, issueList in ctx.issues:
@@ -152,6 +189,16 @@ proc parsePropertiesOption(propsOpt: string): TableRef[string, string] =
     if pair.len == 1: result[pair[0]] = "true"
     else: result[pair[0]] = pair[1]
 
+proc parseExclPropertiesOption(propsOpt: string): TableRef[string, seq[string]] =
+  result = newTable[string, seq[string]]()
+  for propText in propsOpt.split(";"):
+    let pair = propText.split(":", 1)
+    let val =
+      if pair.len == 1: "true"
+      else: pair[1]
+    if result.hasKey(pair[0]): result[pair[0]].add(val)
+    else: result[pair[0]] = @[val]
+
 proc sameDay(a, b: DateTime): bool =
   result = a.year == b.year and a.yearday == b.yearday
 
@@ -162,36 +209,62 @@ proc writeHeader(ctx: CliContext, header: string) =
   stdout.writeLine('~'.repeat(ctx.termWidth))
   stdout.resetAttributes
 
+proc reorder(ctx: CliContext, state: IssueState) =
+
+  # load the issues to make sure the order file contains all issues in the state.
+  ctx.loadIssues(state)
+  discard os.execShellCmd(EDITOR & " '" & (ctx.cfg.tasksDir / $state / "order.txt") & "' </dev/tty >/dev/tty")
+
 proc edit(issue: Issue) =
 
   # Write format comments (to help when editing)
   writeFile(issue.filepath, toStorageFormat(issue, true))
 
-  let editor =
-    if existsEnv("EDITOR"): getEnv("EDITOR")
-    else: "vi"
-
-  discard os.execShellCmd(editor & " " & issue.filepath & " </dev/tty >/dev/tty")
+  discard os.execShellCmd(EDITOR & " '" & issue.filepath & "' </dev/tty >/dev/tty")
 
   try:
     # Try to parse the newly-edited issue to make sure it was successful.
     let editedIssue = loadIssue(issue.filepath)
     editedIssue.store()
-  except:
-    fatal "pit: updated issue is invalid (ignoring edits): \n\t" &
+  except CatchableError:
+    fatal "updated issue is invalid (ignoring edits): \n\t" &
       getCurrentExceptionMsg()
     issue.store()
 
-proc list(ctx: CliContext, filter: Option[IssueFilter], state: Option[IssueState], today, future, verbose: bool) =
+proc list(
+    ctx: CliContext,
+    filter: Option[IssueFilter],
+    states: Option[seq[IssueState]],
+    showToday, showFuture,
+    showHidden = false,
+    verbose: bool) =
 
-  if state.isSome:
-    ctx.loadIssues(state.get)
-    if filter.isSome: ctx.filterIssues(filter.get)
-    stdout.write ctx.formatSection(ctx.issues[state.get], state.get, "", verbose)
+  if states.isSome:
+    trace "listing issues for " & $states.get
+    for state in states.get:
+      ctx.loadIssues(state)
+      if filter.isSome: ctx.filterIssues(filter.get)
+
+      # Show Done for just today if requested
+      if state == Done and showToday:
+        ctx.issues[Done] = ctx.issues[Done].filterIt(
+          it.hasProp("completed") and
+          sameDay(getTime().local, it.getDateTime("completed")))
+
+      stdout.write ctx.formatSection(ctx.issues[state], state, "", verbose)
+    trace "listing complete"
     return
 
-  ctx.loadAllIssues()
-  if filter.isSome: ctx.filterIssues(filter.get)
+  ctx.loadOpenIssues()
+  if filter.isSome:
+    ctx.filterIssues(filter.get)
+    trace "filtered issues"
+
+  let today = showToday and [Current, TodoToday, Pending].anyIt(
+    ctx.issues.hasKey(it) and ctx.issues[it].len > 0)
+
+  let future = showFuture and [Pending, Todo].anyIt(
+    ctx.issues.hasKey(it) and ctx.issues[it].len > 0)
 
   let indent = if today and future: "  " else: ""
 
@@ -199,84 +272,63 @@ proc list(ctx: CliContext, filter: Option[IssueFilter], state: Option[IssueState
   if today:
     if future: ctx.writeHeader("Today")
 
-    for s in [Current, TodoToday]:
+    for s in [Current, TodoToday, Pending]:
       if ctx.issues.hasKey(s) and ctx.issues[s].len > 0:
-        stdout.write ctx.formatSection(ctx.issues[s], s, indent, verbose)
+        let visibleIssues = ctx.issues[s].filterIt(
+          showHidden or
+          not (it.hasProp("hide-until") and
+               it.getDateTime("hide-until") > getTime().local))
 
-    if ctx.issues.hasKey(Done):
-        let doneIssues = ctx.issues[Done].filterIt(
-          it.hasProp("completed") and
-          sameDay(getTime().local, it.getDateTime("completed")))
-        if doneIssues.len > 0:
-          stdout.write ctx.formatSection(doneIssues, Done, indent, verbose)
+        stdout.write ctx.formatSection(visibleIssues, s, indent, verbose)
 
   # Future items
   if future:
     if today: ctx.writeHeader("Future")
 
-    for s in [Pending, Todo]:
+    let futureCategories =
+      if showToday: @[Todo]
+      else: @[Pending, Todo]
+
+    for s in futureCategories:
       if ctx.issues.hasKey(s) and ctx.issues[s].len > 0:
-        stdout.write ctx.formatSection(ctx.issues[s], s, indent, verbose)
+        let visibleIssues = ctx.issues[s].filterIt(
+          showHidden or
+          not (it.hasProp("hide-until") and
+               it.getDateTime("hide-until") > getTime().local))
+
+        stdout.write ctx.formatSection(visibleIssues, s, indent, verbose)
+
+  trace "listing complete"
 
 when isMainModule:
-
  try:
-  let doc = """
-Usage:
-  pit ( new | add) <summary> [<state>] [options]
-  pit list [<listable>] [options]
-  pit ( start | done | pending | todo-today | todo | suspend ) <id>... [options]
-  pit edit <ref>...
-  pit ( delete | rm ) <id>...
 
-Options:
-
-  -h, --help                Print this usage information.
-
-  -p, --properties <props>  Specify properties. Formatted as "key:val;key:val"
-                            When used with the list command this option applies
-                            a filter to the issues listed, only allowing those
-                            which have all of the given properties.
-
-  -c, --context <ctxName>   Shorthand for '-p context:<ctxName>'
-
-  -g, --tags <tags>         Specify tags for an issue.
-
-  -T, --today               Limit to today's issues.
-
-  -F, --future              Limit to future issues.
-
-  -v, --verbose             Show issue details when listing issues.
-
-  -q, --quiet               Suppress verbose output.
-
-  -y, --yes                 Automatically answer "yes" to any prompts.
-
-  -C, --config <cfgFile>    Location of the config file (defaults to $HOME/.pitrc)
-
-  -E, --echo-args           Echo arguments (for debug purposes).
-
-  -d, --tasks-dir           Path to the tasks directory (defaults to the value
-                            configured in the .pitrc file)
-
-  --term-width <width>      Manually set the terminal width to use.
-"""
-
-  logging.addHandler(newConsoleLogger())
+  let consoleLogger = newConsoleLogger(
+    levelThreshold=lvlInfo,
+    fmtStr="pit - $levelname: ")
+  logging.addHandler(consoleLogger)
 
   # Parse arguments
-  let args = docopt(doc, version = PIT_VERSION)
+  let args = docopt(USAGE, version = PIT_VERSION)
+
+  if args["--debug"]:
+    consoleLogger.levelThreshold = lvlDebug
 
   if args["--echo-args"]: stderr.writeLine($args)
 
-  if args["--help"]:
-    stderr.writeLine(doc)
+  if args["help"]:
+    stderr.writeLine(USAGE & "\p")
+    stderr.writeLine(ONLINE_HELP)
     quit()
 
   let ctx = initContext(args)
 
+  trace "context initiated"
+
   var propertiesOption = none(TableRef[string,string])
+  var exclPropsOption = none(TableRef[string,seq[string]])
   var tagsOption = none(seq[string])
+  var exclTagsOption = none(seq[string])
 
   if args["--properties"] or args["--context"]:
 
@@ -288,8 +340,25 @@ Options:
 
     propertiesOption = some(props)
 
+  if args["--excl-properties"] or args["--excl-context"]:
+
+    var exclProps =
+      if args["--excl-properties"]:
+        parseExclPropertiesOption($args["--excl-properties"])
+      else: newTable[string,seq[string]]()
+
+    if args["--excl-context"]:
+      if not exclProps.hasKey("context"): exclProps["context"] = @[]
+      let exclContexts = split($args["--excl-context"], ",")
+      exclProps["context"] = exclProps["context"].concat(exclContexts)
+
+    exclPropsOption = some(exclProps)
+
   if args["--tags"]: tagsOption = some(($args["--tags"]).split(",").mapIt(it.strip))
 
+  if args["--excl-tags"]: exclTagsOption =
+    some(($args["--excl-tags"]).split(",").mapIt(it.strip))
+
   ## Actual command runners
   if args["new"] or args["add"]:
     let state =
@@ -307,12 +376,15 @@ Options:
       summary: $args["<summary>"],
       properties: issueProps,
       tags:
-        if args["--tags"]: ($args["--tags"]).split(",").mapIt(it.strip)
+        if tagsOption.isSome: tagsOption.get
         else: newSeq[string]())
 
-    ctx.tasksDir.store(issue, state)
+    ctx.cfg.tasksDir.store(issue, state)
 
-    stdout.writeLine ctx.formatIssue(issue)
+    stdout.writeLine formatIssue(issue)
+
+  elif args["reorder"]:
+    ctx.reorder(parseEnum[IssueState]($args["<state>"]))
 
   elif args["edit"]:
     for editRef in @(args["<ref>"]):
@@ -320,14 +392,37 @@ Options:
       var stateOption = none(IssueState)
 
       try: stateOption = some(parseEnum[IssueState](editRef))
-      except: discard
+      except CatchableError: discard
 
       if stateOption.isSome:
         let state = stateOption.get
         ctx.loadIssues(state)
         for issue in ctx.issues[state]: edit(issue)
 
-      else: edit(ctx.tasksDir.loadIssueById(editRef))
+      else: edit(ctx.cfg.tasksDir.loadIssueById(editRef))
+
+  elif args["tag"]:
+    if tagsOption.isNone: raise newException(Exception, "no tags given")
+
+    let newTags = tagsOption.get
+
+    for id in @(args["<id>"]):
+      var issue = ctx.cfg.tasksDir.loadIssueById(id)
+      issue.tags = deduplicate(issue.tags & newTags)
+      issue.store()
+
+  elif args["untag"]:
+    let tagsToRemove: seq[string] =
+      if tagsOption.isSome: tagsOption.get
+      else: @[]
+
+    for id in @(args["<id>"]):
+      var issue = ctx.cfg.tasksDir.loadIssueById(id)
+      if tagsToRemove.len > 0:
+        issue.tags = issue.tags.filter(
+          proc (tag: string): bool = not tagsToRemove.anyIt(it == tag))
+      else: issue.tags = @[]
+      issue.store()
 
   elif args["start"] or args["todo-today"] or args["done"] or
        args["pending"] or args["todo"] or args["suspend"]:
@@ -341,27 +436,56 @@ Options:
     elif args["suspend"]: targetState = Dormant
 
     for id in @(args["<id>"]):
-      var issue = ctx.tasksDir.loadIssueById(id)
+      var issue = ctx.cfg.tasksDir.loadIssueById(id)
       if propertiesOption.isSome:
         for k,v in propertiesOption.get:
           issue[k] = v
-      if targetState == Done: issue["completed"] = getTime().local.formatIso8601
-      issue.changeState(ctx.tasksDir, targetState)
+      if targetState == Done:
+        issue["completed"] = getTime().local.formatIso8601
+        if issue.hasProp("recurrence") and issue.getRecurrence.isSome:
+          let nextIssue = ctx.cfg.tasksDir.nextRecurrence(issue.getRecurrence.get, issue)
+          ctx.cfg.tasksDir.store(nextIssue, Todo)
+          info "created the next recurrence:"
+          stdout.writeLine formatIssue(nextIssue)
 
-    if ctx.triggerPtk:
+
+      issue.changeState(ctx.cfg.tasksDir, targetState)
+
+    if ctx.triggerPtk or args["--ptk"]:
       if targetState == Current:
-        let issue = ctx.tasksDir.loadIssueById($(args["<id>"][0]))
-        var cmd = "ptk start "
-        if issue.tags.len > 0: cmd &= "-g \"" & issue.tags.join(",") & "\""
+        let issue = ctx.cfg.tasksDir.loadIssueById($(args["<id>"][0]))
+        var cmd = "ptk start"
+        if issue.tags.len > 0 or issue.hasProp("context"):
+          let tags = concat(
+            issue.tags,
+            if issue.hasProp("context"): @[issue.properties["context"]]
+            else: @[]
+          )
+          cmd &= " -g \"" & tags.join(",") & "\""
+        cmd &= " -n \"pit-id: " & $issue.id & "\""
         cmd &= " \"" & issue.summary & "\""
         discard execShellCmd(cmd)
       elif targetState == Done or targetState == Pending:
         discard execShellCmd("ptk stop")
 
+  elif args["hide-until"]:
+
+    let issue = ctx.cfg.tasksDir.loadIssueById($(args["<id>"]))
+    issue.setDateTime("hide-until", parseDate($args["<date>"]))
+
+    issue.store()
+
+  elif args["delegate"]:
+
+    let issue = ctx.cfg.tasksDir.loadIssueById($(args["<id>"]))
+    issue["delegated-to"] = $args["<delegated-to>"]
+
+    issue.store()
+
   elif args["delete"] or args["rm"]:
     for id in @(args["<id>"]):
 
-      let issue = ctx.tasksDir.loadIssueById(id)
+      let issue = ctx.cfg.tasksDir.loadIssueById(id)
 
       if not args["--yes"]:
         stderr.write("Delete '" & issue.summary & "' (y/n)? ")
@@ -380,27 +504,53 @@ Options:
       filter.properties = propertiesOption.get
       filterOption = some(filter)
 
+    # Add property exclusions (if given)
+    if exclPropsOption.isSome:
+      filter.exclProperties = exclPropsOption.get
+      filterOption = some(filter)
+
+    # If they supplied text matches, add that to the filter.
+    if args["--match"]:
+      filter.summaryMatch = some(re("(?i)" & $args["--match"]))
+      filterOption = some(filter)
+
+    if args["--match-all"]:
+      filter.fullMatch = some(re("(?i)" & $args["--match-all"]))
+      filterOption = some(filter)
+
     # If no "context" property is given, use the default (if we have one)
     if ctx.defaultContext.isSome and not filter.properties.hasKey("context"):
       stderr.writeLine("Limiting to default context: " & ctx.defaultContext.get)
       filter.properties["context"] = ctx.defaultContext.get
       filterOption = some(filter)
 
+    if tagsOption.isSome:
+      filter.hasTags = tagsOption.get
+      filterOption = some(filter)
+
+    if exclTagsOption.isSome:
+      filter.exclTags = exclTagsOption.get
+      filterOption = some(filter)
+
     # Finally, if the "context" is "all", don't filter on context
     if filter.properties.hasKey("context") and
        filter.properties["context"] == "all":
 
       filter.properties.del("context")
+      filter.exclProperties.del("context")
 
     var listContexts = false
-    var stateOption = none(IssueState)
-    var issueIdOption = none(string)
+    var statesOption = none(seq[IssueState])
+    var issueIdsOption = none(seq[string])
 
-    if args["<listable>"]:
-      if $args["<listable>"] == "contexts": listContexts = true
-      else:
-        try: stateOption = some(parseEnum[IssueState]($args["<listable>"]))
-        except: issueIdOption = some($args["<listable>"])
+    if args["contexts"]: listContexts = true
+    elif args["<stateOrId>"]:
+      try:
+        statesOption =
+          some(args["<stateOrId>"].
+          mapIt(parseEnum[IssueState]($it)))
+      except CatchableError:
+        issueIdsOption = some(args["<stateOrId>"].mapIt($it))
 
     # List the known contexts
     if listContexts:
@@ -411,21 +561,76 @@ Options:
           if issue.hasProp("context") and not uniqContexts.contains(issue["context"]):
             uniqContexts.add(issue["context"])
 
-      for c in uniqContexts: stdout.writeLine(c)
+      let maxLen = foldl(uniqContexts,
+        if a.len > b.len: a
+        else: b
+      ).len
+
+      for c in uniqContexts.sorted:
+        stdout.writeLine(c.alignLeft(maxLen+2) & ctx.getIssueContextDisplayName(c))
 
     # List a specific issue
-    elif issueIdOption.isSome:
-      let issue = ctx.tasksDir.loadIssueById(issueIdOption.get)
-      stdout.writeLine ctx.formatIssue(issue)
+    elif issueIdsOption.isSome:
+      for issueId in issueIdsOption.get:
+        let issue = ctx.cfg.tasksDir.loadIssueById(issueId)
+        stdout.writeLine formatIssue(issue)
 
     # List all issues
     else:
+      trace "listing all issues"
       let showBoth = args["--today"] == args["--future"]
-      ctx.list(filterOption, stateOption, showBoth or args["--today"],
-                                          showBoth or args["--future"],
-                                          ctx.verbose)
+      ctx.list(
+        filter = filterOption,
+        states = statesOption,
+        showToday = showBoth or args["--today"],
+        showFuture = showBoth or args["--future"],
+        showHidden = args["--show-hidden"],
+        verbose = ctx.verbose)
 
- except:
-  fatal "pit: " & getCurrentExceptionMsg()
+  elif args["add-binary-property"]:
+    let issue = ctx.cfg.tasksDir.loadIssueById($(args["<id>"]))
+
+    let propIn =
+      if $(args["<propSource>"]) == "-": stdin
+      else: open($(args["<propSource>"]))
+
+    try: issue[$(args["<propName>"])] = encodeAsDataUri(readAll(propIn))
+    finally: close(propIn)
+
+    issue.store()
+
+  elif args["get-binary-property"]:
+    let issue = ctx.cfg.tasksDir.loadIssueById($(args["<id>"]))
+
+    if not issue.hasProp($(args["<propName>"])):
+      raise newException(Exception,
+        "issue " & ($issue.id)[0..<6] & " has no property name '" &
+        $(args["<propName>"]) & "'")
+
+    let propOut =
+      if $(args["<propDest>"]) == "-": stdout
+      else: open($(args["<propDest>"]), fmWrite)
+
+    try: write(propOut, decodeDataUri(issue[$(args["<propName>"])]))
+    finally: close(propOut)
+
+  elif args["show-dupes"]:
+    ctx.loadAllIssues()
+
+    var idsToPaths = newTable[string, var seq[string]]()
+    for (state, issues) in pairs(ctx.issues):
+      for issue in issues:
+        let issueId = $issue.id
+
+        if idsToPaths.hasKey(issueId): idsToPaths[issueId].add(issue.filepath)
+        else: idsToPaths[issueId] = @[issue.filepath]
+
+    for (issueId, issuePaths) in pairs(idsToPaths):
+      if issuePaths.len < 2: continue
+      stdout.writeLine(issueId & ":\p  " & issuePaths.join("\p  ") & "\p\p")
+
+ except CatchableError:
+  fatal getCurrentExceptionMsg()
+  debug getCurrentException().getStackTrace()
   #raise getCurrentException()
   quit(QuitFailure)

src/pit_api.nim

@@ -1,12 +1,11 @@
 ## Personal Issue Tracker API Interface
 ## ====================================
 
-import asyncdispatch, cliutils, docopt, jester, json, logging, sequtils, strutils
+import asyncdispatch, cliutils, docopt, jester, json, logging, options, sequtils, strutils
 import nre except toSeq
 
 import pitpkg/private/libpit
-
-include "pitpkg/version.nim"
+import pitpkg/cliconstants
 
 type
   PitApiCfg* = object
@@ -18,6 +17,20 @@ const TXT = "text/plain"
 
 proc raiseEx(reason: string): void = raise newException(Exception, reason)
 
+template halt(code: HttpCode,
+              headers: RawHeaders,
+              content: string): void =
+  ## Immediately replies with the specified request. This means any further
+  ## code will not be executed after calling this template in the current
+  ## route.
+  bind TCActionSend, newHttpHeaders
+  result[0] = CallbackAction.TCActionSend
+  result[1] = code
+  result[2] = some(headers)
+  result[3] = content
+  result.matched = true
+  break allRoutes
+
 template checkAuth(cfg: PitApiCfg) =
   ## Check this request for authentication and authorization information.
   ## If the request is not authorized, this template sets up the 401 response
@@ -26,10 +39,10 @@ template checkAuth(cfg: PitApiCfg) =
   var authed {.inject.} = false
 
   try:
-    if not request.headers.hasKey("Authorization"):
+    if not headers(request).hasKey("Authorization"):
       raiseEx "No auth token."
 
-    let headerVal = request.headers["Authorization"]
+    let headerVal = headers(request)["Authorization"]
     if not headerVal.startsWith("Bearer "):
       raiseEx "Invalid Authentication type (only 'Bearer' is supported)."
 
@@ -40,30 +53,10 @@ template checkAuth(cfg: PitApiCfg) =
 
   except:
     stderr.writeLine "Auth failed: " & getCurrentExceptionMsg()
-    response.data[0] = CallbackAction.TCActionSend
-    response.data[1] = Http401
-    response.data[2]["WWW-Authenticate"] = "Bearer"
-    response.data[2]["Content-Type"] = TXT
-    response.data[3] = getCurrentExceptionMsg()
-
-proc paramsToArgs(params: StringTableRef): tuple[stripAnsi: bool, args: seq[string]] =
-  result = (false, @[])
-
-  if params.hasKey("color"):
-    if params["color"] != "true":
-      result[0] = true
-
-  for k,v in params:
-    if k == "color": continue
-    elif k.startsWith("arg"): result[1].add(v) # support ?arg1=val1&arg2=val2 -> cmd val1 val2
-    else :
-      result[1].add("--" & k)
-      if v != "true": result[1].add(v) # support things like ?verbose=true -> cmd --verbose
-
-let STRIP_ANSI_REGEX = re"\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]"
-
-proc stripAnsi(str: string): string =
-  return str.replace(STRIP_ANSI_REGEX, "")
+    halt(
+      Http401,
+      @{"Content-Type": TXT},
+      getCurrentExceptionMsg())
 
 proc start*(cfg: PitApiCfg) =
 
@@ -79,20 +72,29 @@ proc start*(cfg: PitApiCfg) =
       resp("pong", TXT)
 
     get "/issues":
-      checkAuth(cfg); if not authed: return true
+      checkAuth(cfg)
 
-      var (stripAnsi, args) = paramsToArgs(request.params)
+      var args = queryParamsToCliArgs(request.params)
       args = @["list"] & args
 
       info "args: \n" & args.join(" ")
       let execResult = execWithOutput("pit", ".", args)
       if execResult[2] != 0: resp(Http500, stripAnsi($execResult[0] & "\n" & $execResult[1]), TXT)
-      else:
-        if stripAnsi: resp(stripAnsi(execResult[0]), TXT)
-        else: resp(execResult[0], TXT)
+      else: resp(stripAnsi(execResult[0]), TXT)
 
     post "/issues":
-      checkAuth(cfg); if not authed: return true
+      checkAuth(cfg)
+
+    get "/issue/@issueId":
+      checkAuth(cfg)
+
+      var args = queryParamsToCliArgs(request.params)
+      args = @["list", @"issueId"] & args
+
+      info "args: \n" & args.join(" ")
+      let execResult = execWithOutput("pit", ".", args)
+      if execResult[2] != 0: resp(Http500, stripAnsi($execResult[0] & "\n" & $execResult[1]), TXT)
+      else: resp(stripAnsi(execResult[0]), TXT)
 
   waitFor(stopFuture)
 

src/pitpkg/cliconstants.nim

@@ -0,0 +1,176 @@
+const PIT_VERSION* = "4.23.4"
+
+const USAGE* = """Usage:
+  pit ( new | add) <summary> [<state>] [options]
+  pit list contexts [options]
+  pit list [<stateOrId>...] [options]
+  pit ( start | done | pending | todo-today | todo | suspend ) <id>... [options]
+  pit edit <ref>... [options]
+  pit tag <id>... [options]
+  pit untag <id>... [options]
+  pit reorder <state> [options]
+  pit delegate <id> <delegated-to> [options]
+  pit hide-until <id> <date> [options]
+  pit ( delete | rm ) <id>... [options]
+  pit add-binary-property <id> <propName> <propSource> [options]
+  pit get-binary-property <id> <propName> <propDest> [options]
+  pit show-dupes
+  pit help [options]
+
+Options:
+
+  -h, --help                Print this usage and help information.
+
+  -p, --properties <props>  Specify properties. Formatted as "key:val;key:val"
+                            When used with the list command this option applies
+                            a filter to the issues listed, only allowing those
+                            which have all of the given properties.
+
+  -P, --excl-properties <props>
+                            When used with the list command, exclude issues
+                            that contain properties with the given value. This
+                            parameter is formatted the same as the --properties
+                            parameter: "key:val;key:val"
+
+  -c, --context <ctx>       Shorthand for '-p context:<ctx>'
+
+  -C, --excl-context <ctx>  Don't show issues from the given context(s).
+                            Multiple contexts can be excluded using a ',' to
+                            separate names. For example: -C ctx1,ctx2
+                            Shorthand for '-P context:<ctx>'
+
+  -g, --tags <tags>         Specify tags for an issue. Tags are specified as a
+                            comma-delimited list. For example: -g tag1,tag2
+
+  -G, --excl-tags <tags>    When used with the list command, exclude issues
+                            that contain any of the provided tags. Tags are
+                            specified as a comma-delimited list.
+                            For example: -G tag1,tag2
+
+  -T, --today               Limit to today's issues.
+
+  -F, --future              Limit to future issues.
+
+  -H, --show-hidden         Show all matching issues, ignoring any 'hide-until'
+                            properties set.
+
+  -m, --match <pattern>     Limit to issues whose summaries match the given
+                            pattern (PCRE regex supported).
+
+  -M, --match-all <pat>     Limit to the issues whose summaries or details
+                            match the given pattern (PCRE regex supported).
+
+  -v, --verbose             Show issue details when listing issues.
+
+  -q, --quiet               Suppress verbose output.
+
+  -y, --yes                 Automatically answer "yes" to any prompts.
+
+  --config <cfgFile>        Location of the config file (defaults to $HOME/.pitrc)
+
+  -E, --echo-args           Echo arguments (for debug purposes).
+
+  -d, --tasks-dir           Path to the tasks directory (defaults to the value
+                            configured in the .pitrc file)
+
+  --term-width <width>      Manually set the terminal width to use.
+
+  --ptk                     Enable PTK integration for this command.
+
+  --debug                   Enable debug-level log output.
+"""
+
+const ONLINE_HELP* = """Issue States:
+
+  PIT organizes issues around their state, which is one of:
+
+    current     - issues actively being worked
+    todo-today  - issues planned for today
+    pending     - issues that are blocked by some third-party
+    done        - issues that have been completely resolved
+    todo        - issues that need to be done in the future
+    dormant     - issues that are low-priority, to be tracked, but hidden
+                  by default
+
+Issue Properties:
+
+  PIT supports adding arbitrary properties to any issue to track any metadata
+  about the issue the user may wish. There are several properties that have
+  special behavior attached to them. They are:
+
+    created
+
+      If present, expected to be an ISO 8601-formatted date that represents the
+      time when the issue was created.
+
+    completed
+
+      If present, expected to be an ISO 8601-formatted date that represents the
+      time when the issue moved to the "done" state.  PIT will add this
+      property automatically when you use the "done" command, and can filter on
+      this value.
+
+    context
+
+      Allows issues to be organized into contexts. The -c option is short-hand
+      for '-p context:<context-name>' and the 'list contexts' command will show
+      all values of 'context' set in existing issues.
+
+    delegated-to
+
+      When an issue now belongs to someone else, but needs to be monitored for
+      completion, this allows you to keep the issue in its current state but
+      note how it has been delegated. When present PIT will prepend this value
+      to the issue summary with an accent color.
+
+    hide-until
+
+      When present, expected to be an ISO 8601-formatted date and used to
+      supress the display of the issue until on or after the given date.
+
+    pending
+
+      When an issue is blocked by a third party, this property can be used to
+      capture details about the dependency When present PIT will display this
+      value after the issue summary.
+
+    recurrence
+
+      When an issue is moved to the "done" state, if the issue has a valid
+      "recurrence" property, PIT will create a new issue and set the
+      "hide-until" property for that new issue depending on the recurrence
+      definition.
+
+      A valid recurrence value has a time value and optionally has an source
+      issue ID. For example:
+
+        every 5 days, 10a544
+
+      The first word, "every", is expected to be either "every" or "after".
+
+      The second portion is expected to be a time period. Supported time units
+      are "hour", "day", "week", "month", an "year", along with the plural
+      forms (e.g. "5 days", "8 hours", etc.).
+
+      The final portion is the source issue ID. This is optional. When a source
+      issue ID is given, the new issue is created as a clone of the given
+      issue. When not given, the issue being closed is used for cloning.
+
+      The "every" and "after" keywords allow the user to choose whether the new
+      issue is created based on the creation time ("every") or the completion
+      time ("after") of the issue being closed based.
+
+      Examples:
+
+        every day
+        every 2 days
+        after 2 days
+        every week
+        after 12 hours
+        every 2 weeks, 10a544
+
+    tags
+
+      If present, expected to be a comma-delimited list of text tags. The -g
+      option is a short-hand for '-p tags:<tags-value>'.
+"""

src/pitpkg/private/libpit.nim

@@ -1,7 +1,10 @@
-import cliutils, docopt, json, logging, options, os, ospaths, sequtils,
-  strutils, tables, times, timeutils, uuids
+import std/[json, logging, options, os, strformat, strutils, tables, times]
+import cliutils, docopt, langutils, uuids, zero_functional
+
+import nre except toSeq
+import timeutils except `>`
+from sequtils import deduplicate, toSeq
 
-from nre import re, match
 type
   Issue* = ref object
     id*: UUID
@@ -9,6 +12,7 @@ type
     summary*, details*: string
     properties*: TableRef[string, string]
     tags*: seq[string]
+    state*: IssueState
 
   IssueState* = enum
     Current = "current",
@@ -19,17 +23,41 @@ type
     Dormant = "dormant"
 
   IssueFilter* = ref object
+    completedRange*: Option[tuple[b, e: DateTime]]
+    fullMatch*, summaryMatch*: Option[Regex]
+    hasTags*: seq[string]
+    exclTags*: seq[string]
     properties*: TableRef[string, string]
-    completedRange*: tuple[b, e: DateTime]
+    exclProperties*: TableRef[string, seq[string]]
 
   PitConfig* = ref object
     tasksDir*: string
     contexts*: TableRef[string, string]
     cfg*: CombinedConfig
 
+  Recurrence* = object
+    cloneId*: Option[string]
+    interval*: TimeInterval
+    isFromCompletion*: bool
+
 const DONE_FOLDER_FORMAT* = "yyyy-MM"
+const ISO8601_MS = "yyyy-MM-dd'T'HH:mm:ss'.'fffzzz"
 
 let ISSUE_FILE_PATTERN = re"[A-Fa-f0-9]{8}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{4}-[A-Fa-f0-9]{12}\.txt"
+let RECURRENCE_PATTERN = re"(every|after) ((\d+) )?((hour|day|week|month|year)s?)(, ([0-9a-fA-F]+))?"
+
+let traceStartTime = cpuTime()
+var lastTraceTime = traceStartTime
+
+proc trace*(msg: string, diffFromLast = false) =
+  let curTraceTime = cpuTime()
+
+  if diffFromLast:
+    debug &"{(curTraceTime - lastTraceTime) * 1000:6.2f}ms {msg}"
+  else:
+    debug &"{cpuTime() - traceStartTime:08.4f} {msg}"
+
+  lastTraceTime = curTraceTime
 
 proc displayName*(s: IssueState): string =
   case s
@@ -47,6 +75,7 @@ proc `[]`*(issue: Issue, key: string): string =
 proc `[]=`*(issue: Issue, key: string, value: string) =
   issue.properties[key] = value
 
+## Issue property accessors
 proc hasProp*(issue: Issue, key: string): bool =
   return issue.properties.hasKey(key)
 
@@ -60,24 +89,89 @@ proc getDateTime*(issue: Issue, key: string, default: DateTime): DateTime =
 proc setDateTime*(issue: Issue, key: string, dt: DateTime) =
   issue.properties[key] = dt.formatIso8601
 
+proc getRecurrence*(issue: Issue): Option[Recurrence] =
+  if not issue.hasProp("recurrence"): return none[Recurrence]()
+
+  let m = issue["recurrence"].match(RECURRENCE_PATTERN)
+
+  if not m.isSome:
+    warn "not a valid recurrence value: '" & issue["recurrence"] & "'"
+    return none[Recurrence]()
+
+  let c = nre.toSeq(m.get.captures)
+  let timeVal = if c[2].isSome: c[2].get.parseInt
+                else: 1
+  return some(Recurrence(
+    isFromCompletion: c[0].get == "after",
+    interval:
+      case c[4].get:
+        of "hour": hours(timeVal)
+        of "day": days(timeVal)
+        of "week": weeks(timeVal)
+        of "month": months(timeVal)
+        of "year": years(timeVal)
+        else: weeks(1),
+    cloneId: c[6]))
+
+## Issue filtering
 proc initFilter*(): IssueFilter =
   result = IssueFilter(
-    properties: newTable[string,string](),
-    completedRange: (fromUnix(0).local, fromUnix(253400659199).local))
+    completedRange: none(tuple[b, e: DateTime]),
+    fullMatch: none(Regex),
+    summaryMatch: none(Regex),
+    hasTags: @[],
+    exclTags: @[],
+    properties: newTable[string, string](),
+    exclProperties: newTable[string,seq[string]]())
 
-proc initFilter*(props: TableRef[string, string]): IssueFilter =
+proc propsFilter*(props: TableRef[string, string]): IssueFilter =
   if isNil(props):
     raise newException(ValueError,
       "cannot initialize property filter without properties")
 
-  result = IssueFilter(
-    properties: props,
-    completedRange: (fromUnix(0).local, fromUnix(253400659199).local))
+  result = initFilter()
+  result.properties = props
 
-proc initFilter*(range: tuple[b, e: DateTime]): IssueFilter =
-  result = IssueFilter(
-    properties: newTable[string, string](),
-    completedRange: range)
+proc dateFilter*(range: tuple[b, e: DateTime]): IssueFilter =
+  result = initFilter()
+  result.completedRange = some(range)
+
+proc summaryMatchFilter*(pattern: string): IssueFilter =
+  result = initFilter()
+  result.summaryMatch = some(re("(?i)" & pattern))
+
+proc fullMatchFilter*(pattern: string): IssueFilter =
+  result = initFilter()
+  result.fullMatch = some(re("(?i)" & pattern))
+
+proc hasTagsFilter*(tags: seq[string]): IssueFilter =
+  result = initFilter()
+  result.hasTags = tags
+
+proc groupBy*(issues: seq[Issue], propertyKey: string): TableRef[string, seq[Issue]] =
+  result = newTable[string, seq[Issue]]()
+  for i in issues:
+    let key = if i.hasProp(propertyKey): i[propertyKey] else: ""
+    if not result.hasKey(key): result[key] = newSeq[Issue]()
+    result[key].add(i)
+
+
+## Parse and format dates
+const DATE_FORMATS = [
+  "MM/dd",
+  "MM-dd",
+  "yyyy-MM-dd",
+  "yyyy/MM/dd",
+  "yyyy-MM-dd'T'hh:mm:ss"
+]
+proc parseDate*(d: string): DateTime =
+  var errMsg = ""
+  for df in DATE_FORMATS:
+    try: return d.parse(df)
+    except CatchableError:
+      errMsg &= "\n\tTried " & df & " with " & d
+      continue
+  raise newException(ValueError, "Unable to parse input as a date: " & d & errMsg)
 
 ## Parse and format issues
 proc fromStorageFormat*(id: string, issueTxt: string): Issue =
@@ -102,7 +196,7 @@ proc fromStorageFormat*(id: string, issueTxt: string): Issue =
 
     of ReadingProps:
       # Ignore empty lines
-      if line.isNilOrWhitespace: continue
+      if line.isEmptyOrWhitespace: continue
 
       # Look for the sentinal to start parsing as detail lines
       if line == "--------":
@@ -110,12 +204,13 @@ proc fromStorageFormat*(id: string, issueTxt: string): Issue =
         continue
 
 
-      let parts = line.split({':'}, 1).mapIt(it.strip())
+      let parts = line.split({':'}, 1) --> map(it.strip())
       if parts.len != 2:
         raise newException(ValueError, "unable to parse property line: " & line)
 
       # Take care of special properties: `tags`
-      if parts[0] == "tags": result.tags = parts[1].split({','}).mapIt(it.strip())
+      if parts[0] == "tags":
+        result.tags = parts[1].split({','}) --> map(it.strip())
       else: result[parts[0]] = parts[1]
 
     of ReadingDetails:
@@ -125,12 +220,18 @@ proc fromStorageFormat*(id: string, issueTxt: string): Issue =
 
 proc toStorageFormat*(issue: Issue, withComments = false): string =
   var lines: seq[string] = @[]
+
   if withComments: lines.add("# Summary (one line):")
   lines.add(issue.summary)
+
   if withComments: lines.add("# Properties (\"key:value\" per line):")
-  for key, val in issue.properties: lines.add(key & ": " & val)
+  issue.properties["last-updated"] = now().format(ISO8601_MS)
+  for key, val in issue.properties:
+    if not val.isEmptyOrWhitespace: lines.add(key & ": " & val)
+
   if issue.tags.len > 0: lines.add("tags: " & issue.tags.join(","))
-  if not isNilOrWhitespace(issue.details) or withComments:
+
+  if not isEmptyOrWhitespace(issue.details) or withComments:
     if withComments: lines.add("# Details go below the \"--------\"")
     lines.add("--------")
     lines.add(issue.details)
@@ -142,6 +243,11 @@ proc loadIssue*(filePath: string): Issue =
   result = fromStorageFormat(splitFile(filePath).name, readFile(filePath))
   result.filepath = filePath
 
+  let parentDirName = filePath.splitFile().dir.splitFile().name
+  let issueState = IssueState.items.toSeq --> find($it == parentDirName)
+  if issueState.isSome: result.state = issueState.get
+  else: result.state = IssueState.Done
+
 proc loadIssueById*(tasksDir, id: string): Issue =
   for path in walkDirRec(tasksDir):
     if path.splitFile.name.startsWith(id):
@@ -149,6 +255,7 @@ proc loadIssueById*(tasksDir, id: string): Issue =
   raise newException(KeyError, "cannot find issue for id: " & id)
 
 proc store*(issue: Issue, withComments = false) =
+  discard existsOrCreateDir(issue.filePath.parentDir)
   writeFile(issue.filepath, toStorageFormat(issue, withComments))
 
 proc store*(tasksDir: string, issue: Issue, state: IssueState, withComments = false) =
@@ -160,87 +267,186 @@ proc store*(tasksDir: string, issue: Issue, state: IssueState, withComments = fa
   else:
     issue.filepath = stateDir / filename
 
-  issue.store()
+  issue.store(withComments)
+
+proc storeOrder*(issues: seq[Issue], path: string) =
+  var orderLines = newSeq[string]()
+
+  for context, issues in issues.groupBy("context"):
+    orderLines.add("> " & context)
+    for issue in issues: orderLines.add($issue.id & " " & issue.summary)
+    orderLines.add("")
+
+  let orderFile = path / "order.txt"
+  orderFile.writeFile(orderLines.join("\n"))
 
 proc loadIssues*(path: string): seq[Issue] =
-  result = @[]
+  let orderFile = path / "order.txt"
+
+  trace "loading issues under " & path
+
+  let orderedIds =
+    if fileExists(orderFile):
+      (orderFile.lines.toSeq -->
+        map(it.split(' ')[0]).
+        filter(not it.startsWith("> ") and not it.isEmptyOrWhitespace)).
+        deduplicate()
+    else: newSeq[string]()
+
+  type TaggedIssue = tuple[issue: Issue, ordered: bool]
+  var unorderedIssues: seq[TaggedIssue] = @[]
+
   for path in walkDirRec(path):
     if extractFilename(path).match(ISSUE_FILE_PATTERN).isSome():
-      result.add(loadIssue(path))
+      unorderedIssues.add((loadIssue(path), false))
+
+  trace "loaded " & $unorderedIssues.len & " issues", true
+  result = @[]
+
+  # Add all ordered issues in order
+  for id in orderedIds:
+    let idx = unorderedIssues.indexOf(($it.issue.id).startsWith(id))
+    if idx > 0:
+      result.add(unorderedIssues[idx].issue)
+      unorderedIssues[idx].ordered = true
+
+  # Add all remaining, unordered issues in the order they were loaded
+  for taggedIssue in unorderedIssues:
+    if taggedIssue.ordered: continue
+    result.add(taggedIssue.issue)
+
+  trace "ordered the loaded issues", true
+
+  # Finally, save current order
+  result.storeOrder(path)
+
+proc loadIssues*(tasksDir: string, state: IssueState): seq[Issue] =
+  loadIssues(tasksDir / $state)
+
+proc loadAllIssues*(tasksDir: string): TableRef[IssueState, seq[Issue]] =
+  result = newTable[IssueState, seq[Issue]]()
+  for state in IssueState: result[state] = tasksDir.loadIssues(state)
 
 proc changeState*(issue: Issue, tasksDir: string, newState: IssueState) =
   let oldFilepath = issue.filepath
   if newState == Done: issue.setDateTime("completed", getTime().local)
   tasksDir.store(issue, newState)
-  removeFile(oldFilepath)
+  if oldFilePath != issue.filepath: removeFile(oldFilepath)
+  issue.state = newState
 
 proc delete*(issue: Issue) = removeFile(issue.filepath)
 
-## Utilities for working with issue collections.
-proc groupBy*(issues: seq[Issue], propertyKey: string): TableRef[string, seq[Issue]] =
-  result = newTable[string, seq[Issue]]()
-  for i in issues:
-    let key = if i.hasProp(propertyKey): i[propertyKey] else: ""
-    if not result.hasKey(key): result[key] = newSeq[Issue]()
-    result[key].add(i)
+proc nextRecurrence*(tasksDir: string, rec: Recurrence, defaultIssue: Issue): Issue =
+  let baseIssue = if rec.cloneId.isSome: tasksDir.loadIssueById(rec.cloneId.get)
+                  else: defaultIssue
 
+  let newProps = newTable[string,string]()
+  for k, v in baseIssue.properties:
+    if k != "completed": newProps[k] = v
+  newProps["prev-recurrence"] = $baseIssue.id
+
+  result = Issue(
+    id: genUUID(),
+    state: baseIssue.state,
+    summary: baseIssue.summary,
+    properties: newProps,
+    tags: baseIssue.tags)
+
+  let now = getTime().local
+
+  let startDate =
+    if rec.isFromCompletion:
+      if baseIssue.hasProp("completed"): baseIssue.getDateTime("completed")
+      else: now
+    else:
+      if baseIssue.hasProp("created"): baseIssue.getDateTime("created")
+      else: now
+
+  # walk the calendar until the next recurrence that is after the current time.
+  var nextTime = startDate + rec.interval
+  while now > nextTime: nextTime += rec.interval
+
+  result.setDateTime("hide-until", nextTime)
+
+## Utilities for working with issue collections.
 proc filter*(issues: seq[Issue], filter: IssueFilter): seq[Issue] =
-  result = issues
+  var f: seq[Issue] = issues
 
   for k,v in filter.properties:
-    result = result.filterIt(it.hasProp(k) and it[k] == v)
+    f = f --> filter(it.hasProp(k) and it[k] == v)
 
-  result = result.filterIt(not it.hasProp("completed") or
-           it.getDateTime("completed").between(
-             filter.completedRange.b,
-             filter.completedRange.e))
+  for k,v in filter.exclProperties:
+    f = f --> filter(not (it.hasProp(k) and v.contains(it[k])))
+
+  if filter.completedRange.isSome:
+    let range = filter.completedRange.get
+    f = f --> filter(
+             not it.hasProp("completed") or
+             it.getDateTime("completed").between(range.b, range.e))
+
+  if filter.summaryMatch.isSome:
+    let p = filter.summaryMatch.get
+    f = f --> filter(it.summary.find(p).isSome)
+
+  if filter.fullMatch.isSome:
+    let p = filter.fullMatch.get
+    f = f -->
+      filter(it.summary.find(p).isSome or it.details.find(p).isSome)
+
+  for tagLent in filter.hasTags:
+    let tag = tagLent
+    f = f --> filter(it.tags.find(tag) >= 0)
+
+  for exclTagLent in filter.exclTags:
+    let exclTag = exclTagLent
+    f = f --> filter(it.tags.find(exclTag) < 0)
+
+  return f # not using result because zero_functional doesn't play nice with it
+
+proc find*(
+    issues: TableRef[IssueState, seq[Issue]],
+    filter: IssueFilter
+  ): seq[Issue] =
+
+  result = @[]
+  for stateIssues in issues.values: result &= stateIssues.filter(filter)
 
 ### Configuration utilities
 proc loadConfig*(args: Table[string, Value] = initTable[string, Value]()): PitConfig =
-  let pitrcLocations = @[
-    if args["--config"]: $args["--config"] else: "",
-    ".pitrc", $getEnv("PITRC"), $getEnv("HOME") & "/.pitrc"]
+  var pitrcFilename: string
 
-  var pitrcFilename: string =
-    foldl(pitrcLocations, if len(a) > 0: a elif existsFile(b): b else: "")
-
-  if not existsFile(pitrcFilename):
-    warn "pit: could not find .pitrc file: " & pitrcFilename
-    if isNilOrWhitespace(pitrcFilename):
+  try:
+    pitrcFilename = findConfigFile(".pitrc",
+      if args["--config"]: @[$args["--config"]] else: @[])
+  except ValueError:
+    warn "could not find .pitrc file: " & pitrcFilename
+    if isEmptyOrWhitespace(pitrcFilename):
       pitrcFilename = $getEnv("HOME") & "/.pitrc"
     var cfgFile: File
     try:
       cfgFile = open(pitrcFilename, fmWrite)
       cfgFile.write("{\"tasksDir\": \"/path/to/tasks\"}")
-    except: warn "pit: could not write default .pitrc to " & pitrcFilename
+    except CatchableError: warn "could not write default .pitrc to " & pitrcFilename
     finally: close(cfgFile)
 
-  var cfgJson: JsonNode
-  try: cfgJson = parseFile(pitrcFilename)
-  except: raise newException(IOError,
-    "unable to read config file: " & pitrcFilename &
-    "\x0D\x0A" & getCurrentExceptionMsg())
-
-  let cfg = CombinedConfig(docopt: args, json: cfgJson)
+  debug "loading config from '$#'" % [pitrcFilename]
+  let cfg = initCombinedConfig(pitrcFilename, args)
 
   result = PitConfig(
     cfg: cfg,
     contexts: newTable[string,string](),
     tasksDir: cfg.getVal("tasks-dir", ""))
 
-  if cfgJson.hasKey("contexts"):
-    for k, v in cfgJson["contexts"]:
-      result.contexts[k] = v.getStr()
+  for k, v in cfg.getJson("contexts", newJObject()):
+    result.contexts[k] = v.getStr()
 
-  if isNilOrWhitespace(result.tasksDir):
+  if isEmptyOrWhitespace(result.tasksDir):
     raise newException(Exception, "no tasks directory configured")
 
-  if not existsDir(result.tasksDir):
+  if not dirExists(result.tasksDir):
     raise newException(Exception, "cannot find tasks dir: " & result.tasksDir)
 
   # Create our tasks directory structure if needed
   for s in IssueState:
-    if not existsDir(result.tasksDir / $s):
+    if not dirExists(result.tasksDir / $s):
       (result.tasksDir / $s).createDir
-
-

src/pitpkg/version.nim

@@ -1 +0,0 @@
-const PIT_VERSION = "4.3.0"