Bump version in prep for release.

WIP documentation
- The documentation is cluttered enough as it is with the large number of procedures supporting vCard 3 and 4. Split common out into the publicly exposed bits and the private internals. This makes it obvious which common functionality a client can expect to have exposed on the main vcard module. - Add documentation (WIP) on the vcard3 module.
2023-05-03 07:11:32 -05:00 · 2023-05-03 02:16:18 -05:00 · 2023-05-03 01:05:02 -05:00 · 2023-05-03 00:29:55 -05:00 · 2023-05-03 00:28:26 -05:00 · 2023-05-02 22:37:23 -05:00
21 changed files with 3678 additions and 1374 deletions
--- a/.gitignore
+++ b/.gitignore
@ -1,3 +1,7 @@
+tests/*
+!tests/*.*
+
+
+bin/
+doc/
 *.sw?
-tests/tlexer
-tests/tvcard3
--- a/31
+++ b/31
@ -0,0 +1,31 @@
+# Make does not offer a recursive wildcard function, so here's one:
+rwildcard=$(wildcard $1$2) $(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2))
+
+SOURCES=$(call rwildcard,src/,*.nim)
+TEST_SOURCES=$(wildcard tests/*.nim)
+TESTS=$(patsubst %.nim,bin/%,$(TEST_SOURCES))
+
+.PHONY: build
+build: test docs
+
+doc/vcard/vcard.html: $(SOURCES)
+	nim doc --project --outdir:doc/vcard src/vcard.nim
+
+.PHONY: doc
+docs: doc/vcard/vcard.html
+
+.PHONY: test
+test:
+	#@for t in $(TESTS); do $$t; done
+	nimble --warning:BareExcept:off test
+
+.PHONY: install
+install: test
+	nimble install
+
+diagrams: doc/vcard3.mmd
+	mmdc -i doc/vcard3.mmd -o doc/vcard3.png
+
+# Target allowing for running individual tests.
+bin/tests/%: tests/%.nim $(SOURCES)
+	nim --outdir:bin/tests --hints:off --warning:BareExcept:off c -r $(patsubst bin/%,%.nim,$@)
--- a/README.md
+++ b/README.md
@ -22,30 +22,7 @@ TEL;TYPE=CELL:+1 (555) 123-4567
 END:VCARD
 ```

-```nim
-import vcard
-
-# Reading in an existing vcard
-let vcards = parseVCard3File("jack.vcf")
-assert vcards.len == 1
-let vcAllen = vcards[0]
-
-assert vcAllen.email.len == 2
-assert vcAllen.email[0].value == "allen@fosters.test"
-assert vcAllen.n.first == "Jack"
-
-
-# Creating a new VCard
-var vcSusan: VCard3
-vcSusan.add(
-  newVC3_N(given = "Susan", family = "Foster"),
-  newVC3_Email(value = "susan@fosters.test", emailType = @["PREF", $etInternet),
-  newVC3_Tel(
-    value = "+1 (555) 444-3889",
-    telType = @[$ttHome, $ttCell, $ttVoice, $ttMsg])
-)
-writeFile("susan.vcf", $vcSusan)
-```
+https://github.com/jdbernard/nim-vcard/blob/4839ff64a8e6da1ad4803adbd71c0a53cae81c4e/examples/simple.nim#L1-L22

 ## Future Goals

@ -55,9 +32,9 @@ writeFile("susan.vcf", $vcSusan)

 *Need to clean up and organize*

-Run `tlexer` tests in gdb:
+Run `tvcard3` tests in gdb:

 ```sh
 $ cd tests
-$ nim --debuginfo --linedir:on c tlexer
-$ gdb --tui tlexer
+$ nim --debuginfo --linedir:on c tvcard3
+$ gdb --tui tvcard3
--- a/doc/rfc6868.txt
+++ b/doc/rfc6868.txt
@ -0,0 +1,395 @@
+
+
+
+
+
+
+Internet Engineering Task Force (IETF)                          C. Daboo
+Request for Comments: 6868                                         Apple
+Updates: 5545, 6321, 6350, 6351                            February 2013
+Category: Standards Track
+ISSN: 2070-1721
+
+
+            Parameter Value Encoding in iCalendar and vCard
+
+Abstract
+
+   This specification updates the data formats for iCalendar (RFC 5545)
+   and vCard (RFC 6350) to allow parameter values to include certain
+   characters forbidden by the existing specifications.
+
+Status of This Memo
+
+   This is an Internet Standards Track document.
+
+   This document is a product of the Internet Engineering Task Force
+   (IETF).  It represents the consensus of the IETF community.  It has
+   received public review and has been approved for publication by the
+   Internet Engineering Steering Group (IESG).  Further information on
+   Internet Standards is available in Section 2 of RFC 5741.
+
+   Information about the current status of this document, any errata,
+   and how to provide feedback on it may be obtained at
+   http://www.rfc-editor.org/info/rfc6868.
+
+Copyright Notice
+
+   Copyright (c) 2013 IETF Trust and the persons identified as the
+   document authors.  All rights reserved.
+
+   This document is subject to BCP 78 and the IETF Trust's Legal
+   Provisions Relating to IETF Documents
+   (http://trustee.ietf.org/license-info) in effect on the date of
+   publication of this document.  Please review these documents
+   carefully, as they describe your rights and restrictions with respect
+   to this document.  Code Components extracted from this document must
+   include Simplified BSD License text as described in Section 4.e of
+   the Trust Legal Provisions and are provided without warranty as
+   described in the Simplified BSD License.
+
+
+
+
+
+
+
+
+Daboo                        Standards Track                    [Page 1]
+
+RFC 6868                   Parameter Encoding              February 2013
+
+
+Table of Contents
+
+   1. Introduction ....................................................2
+   2. Conventions Used in This Document ...............................2
+   3. Parameter Value Encoding Scheme .................................3
+      3.1. iCalendar Example ..........................................4
+      3.2. vCard Example ..............................................4
+   4. Security Considerations .........................................4
+   5. Acknowledgments .................................................4
+   6. Normative References ............................................5
+   Appendix A. Choice of Quoting Mechanism ............................6
+
+1.  Introduction
+
+   The iCalendar [RFC5545] specification defines a standard way to
+   describe calendar data.  The vCard [RFC6350] specification defines a
+   standard way to describe contact data.  Both of these use a similar
+   text-based data format.  Each iCalendar and vCard data object can
+   include "properties" that have "parameters" and a "value".  The value
+   of a "parameter" is typically a token or URI value, but a "generic"
+   text value is also allowed.  However, the syntax rules for both
+   iCalendar and vCard prevent the use of a double-quote character or
+   control characters in such values, though double-quote characters and
+   some subset of control characters are allowed in the actual property
+   values.
+
+   As more and more extensions are being developed for these data
+   formats, there is a need to allow at least double-quotes and line
+   feeds to be included in parameter values.  The \-escaping mechanism
+   used for property text values is not defined for use with parameter
+   values and cannot be easily used in a backwards-compatible manner.
+   This specification defines a new character escaping mechanism,
+   compatible with existing parsers and chosen to minimize any impact on
+   existing data.
+
+2.  Conventions Used in This Document
+
+   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
+   "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
+   "OPTIONAL" in this document are to be interpreted as described in
+   [RFC2119].
+
+
+
+
+
+
+
+
+
+
+Daboo                        Standards Track                    [Page 2]
+
+RFC 6868                   Parameter Encoding              February 2013
+
+
+3.  Parameter Value Encoding Scheme
+
+   This specification defines the ^ character (U+005E -- Circumflex
+   Accent) as an escape character in parameter values whose value type
+   is defined using the "param-value" syntax element (Section 3.1 of
+   iCalendar [RFC5545] and Section 3.3 of vCard [RFC6350]).  The
+   ^-escaping mechanism can be used when the value is either unquoted or
+   quoted (i.e., whether or not the value is surrounded by double-
+   quotes).
+
+   When generating iCalendar or vCard parameter values, the following
+   apply:
+
+   o  formatted text line breaks are encoded into ^n (U+005E, U+006E)
+
+   o  the ^ character (U+005E) is encoded into ^^ (U+005E, U+005E)
+
+   o  the " character (U+0022) is encoded into ^' (U+005E, U+0027)
+
+   When parsing iCalendar or vCard parameter values, the following
+   apply:
+
+   o  the character sequence ^n (U+005E, U+006E) is decoded into an
+      appropriate formatted line break according to the type of system
+      being used
+
+   o  the character sequence ^^ (U+005E, U+005E) is decoded into the ^
+      character (U+005E)
+
+   o  the character sequence ^' (U+005E, U+0027) is decoded into the "
+      character (U+0022)
+
+   o  if a ^ (U+005E) character is followed by any character other than
+      the ones above, parsers MUST leave both the ^ and the following
+      character in place
+
+   When converting between iCalendar and vCard text-based data formats
+   and alternative data-format representations such as XML (as described
+   in [RFC6321] and [RFC6351], respectively), implementations MUST
+   ensure that parameter value escape sequences are generated correctly
+   in the text-based format and are decoded when the parameter values
+   appear in the alternate data formats.
+
+
+
+
+
+
+
+
+
+Daboo                        Standards Track                    [Page 3]
+
+RFC 6868                   Parameter Encoding              February 2013
+
+
+3.1.  iCalendar Example
+
+   The following example is an "ATTENDEE" property with a "CN" parameter
+   whose value includes two double-quote characters.  The parameter
+   value is not quoted, as there are no characters in the value that
+   would trigger quoting as required by iCalendar.
+
+   ATTENDEE;CN=George Herman ^'Babe^' Ruth:mailto:babe@example.com
+
+   The unescaped parameter value is
+
+   George Herman "Babe" Ruth
+
+3.2.  vCard Example
+
+   The following example is a "GEO" property with an "X-ADDRESS"
+   parameter whose value includes several line feed characters.  The
+   parameter value is also quoted, since it contains a comma, which
+   triggers quoting as required by vCard.
+
+   GEO;X-ADDRESS="Pittsburgh Pirates^n115 Federal St^nPitt
+    sburgh, PA 15212":geo:40.446816,-80.00566
+
+   The unescaped parameter value (where each line is terminated by a
+   line break character sequence) is
+
+   Pittsburgh Pirates
+   115 Federal St
+   Pittsburgh, PA 15212
+
+4.  Security Considerations
+
+   There are no additional security issues beyond those of iCalendar
+   [RFC5545] and vCard [RFC6350].
+
+5.  Acknowledgments
+
+   Thanks to Michael Angstadt, Tim Bray, Mike Douglass, Barry Leiba,
+   Simon Perreault, and Pete Resnick for feedback on this specification.
+
+
+
+
+
+
+
+
+
+
+
+
+Daboo                        Standards Track                    [Page 4]
+
+RFC 6868                   Parameter Encoding              February 2013
+
+
+6.  Normative References
+
+   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
+              Requirement Levels", BCP 14, RFC 2119, March 1997.
+
+   [RFC5545]  Desruisseaux, B., "Internet Calendaring and Scheduling
+              Core Object Specification (iCalendar)", RFC 5545,
+              September 2009.
+
+   [RFC6321]  Daboo, C., Douglass, M., and S. Lees, "xCal: The XML
+              Format for iCalendar", RFC 6321, August 2011.
+
+   [RFC6350]  Perreault, S., "vCard Format Specification", RFC 6350,
+              August 2011.
+
+   [RFC6351]  Perreault, S., "xCard: vCard XML Representation",
+              RFC 6351, August 2011.
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Daboo                        Standards Track                    [Page 5]
+
+RFC 6868                   Parameter Encoding              February 2013
+
+
+Appendix A.  Choice of Quoting Mechanism
+
+   Having recognized the need for escaping parameter values, the
+   question is what mechanism to use?  One obvious choice would be to
+   adopt the \-escaping used for property values.  However, that could
+   not be used as-is, because it escapes a double-quote as the sequence
+   of \ followed by double-quote.  Consider what the example in
+   Section 3.1 might look like using \-escaping:
+
+   ATTENDEE;CN="George Herman \"Babe\" Ruth":mailto:babe@example.com
+
+   Existing iCalendar/vCard parsers know nothing about escape sequences
+   in parameters.  So they would parse the parameter value as:
+
+   George Herman \
+
+   i.e., the text between the first and second occurrence of a double-
+   quote.  However, the text after the second double-quote ought to be
+   either a : or a ; (to delimit the parameter value from the following
+   parameter or property) but is not, so the parser could legitimately
+   throw an error at that point because the data is syntactically
+   invalid.  Thus, for backwards-compatibility reasons, a double-quote
+   cannot be escaped using a sequence that itself includes a double-
+   quote, and hence the choice of using a single-quote in this
+   specification.
+
+   Another option would be to use a form of \-escaping modified for use
+   in parameter values only.  However, some incorrect, non-interoperable
+   use of \ in parameter values has been observed, and thus it is best
+   to steer clear of that to achieve guaranteed, reliable
+   interoperability.  Also, given that double-quote gets changed to
+   single-quote in the escape sequence for a parameter, but not for a
+   value, it is better to not give the impression that the same escape
+   mechanism (and thus code) can be used for both (which could lead to
+   other issues, such as an implementation incorrectly escaping a ; as
+   \; as opposed to quoting the parameter value).
+
+   The choice of ^ as the escape character was made based on the
+   requirement that an ASCII symbol (non-alphanumeric character) be
+   used, and it ought to be one least likely to be found in existing
+   data.
+
+
+
+
+
+
+
+
+
+
+Daboo                        Standards Track                    [Page 6]
+
+RFC 6868                   Parameter Encoding              February 2013
+
+
+Author's Address
+
+   Cyrus Daboo
+   Apple Inc.
+   1 Infinite Loop
+   Cupertino, CA  95014
+   USA
+
+   EMail: cyrus@daboo.name
+   URI:   http://www.apple.com/
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+Daboo                        Standards Track                    [Page 7]
+
--- a/examples/jack.vcf
+++ b/examples/jack.vcf
@ -0,0 +1,10 @@
+BEGIN:VCARD
+VERSION:3.0
+UID: 5db6f100-e2d6-4e8d-951f-d920586bc069
+N:Foster;Jack;Allen;;
+FN:Allen Foster
+REV:20230408T122102Z
+EMAIL;TYPE=home;TYPE=pref:allen@fosters.test
+EMAIL;TYPE=work:jack.foster@company.test
+TEL;TYPE=CELL:+1 (555) 123-4567
+END:VCARD
--- a/examples/simple.nim
+++ b/examples/simple.nim
@ -0,0 +1,22 @@
+import vcard
+
+# Reading in an existing vcard
+let vcards = parseVCard3File("jack.vcf")
+assert vcards.len == 1
+let vcAllen = vcards[0]
+
+assert vcAllen.email.len == 2
+assert vcAllen.email[0].value == "allen@fosters.test"
+assert vcAllen.n.given[0] == "Jack"
+
+# Creating a new VCard
+var vcSusan: VCard3
+vcSusan.add(@[
+  newVC3_N(given = @["Susan"], family = @["Foster"]),
+  newVC3_Email(value = "susan@fosters.test", emailType = @["PREF",
+  $etInternet]),
+  newVC3_Tel(
+    value = "+1 (555) 444-3889",
+    telType = @[$ttHome, $ttCell, $ttVoice, $ttMsg])
+])
+writeFile("susan.vcf", $vcSusan)
--- a/src/vcard.nim
+++ b/src/vcard.nim
@ -1,3 +1,76 @@
-import vcard/vcard3
+# vCard 3.0 and 4.0 Nim implementation
+# © 2022 Jonathan Bernard

-export vcard3
+## The `vcard` module implements a high-performance vCard parser for both
+## versions 3.0 (defined by RFCs 2425_ and  2426_) and 4.0 (defined by RFC
+## 6350_)
+##
+## .. _2425: https://tools.ietf.org/html/rfc2425
+## .. _2426: https://tools.ietf.org/html/rfc2426
+## .. _6350: https://tools.ietf.org/html/rfc6350
+import std/[streams, unicode]
+
+import ./vcard/private/[internals, lexer]
+import ./vcard/[common, vcard3, vcard4]
+
+export vcard3, vcard4
+export common.VC_Param,
+       common.VC_XParam,
+       common.VCard,
+       common.VCardParsingError,
+       common.VCardVersion,
+       common.allPropsOfType,
+       common.getMultipleValues,
+       common.getSingleValue
+
+proc add*[T](vc: VCard, content: varargs[T]): void =
+  if vc.parsedVersion == VCardV3: add(cast[VCard3](vc), content)
+  else: add(cast[VCard4](vc), content)
+
+proc readVCard*(p: var VCardParser): VCard =
+  # Read the preamble
+  discard p.readGroup
+  p.expect("begin:vcard" & CRLF)
+
+  # Look for the version tag
+  p.setBookmark
+  discard p.readGroup
+  if p.isNext("version:4.0"):
+    result = VCard4()
+    result.parsedVersion = VCardV4
+  else:
+    result = VCard3()
+    result.parsedVersion = VCardV3
+  p.returnToBookmark
+
+  # VCard3 3.0 allows arbitrarily many empty lines after BEGIN and END
+  if result.parsedVersion == VCardV3:
+    while (p.skip(CRLF, true)): discard
+    for content in vcard3.parseContentLines(p): result.add(content)
+    while (p.skip(CRLF, true)): discard
+
+  else:
+    for content in vcard4.parseContentLines(p): result.add(content)
+
+  if result.parsedVersion == VCardV3:
+    while (p.skip(CRLF, true)): discard
+
+proc initVCardParser*(input: Stream, filename = "input"): VCardParser =
+  result.filename = filename
+  lexer.open(result, input)
+
+proc initVCardParser*(content: string, filename = "input"): VCardParser =
+  initVCardParser(newStringStream(content), filename)
+
+proc initVCardParserFromFile*(filepath: string): VCardParser =
+  initVCardParser(newFileStream(filepath, fmRead), filepath)
+
+proc parseVCards*(input: Stream, filename = "input"): seq[VCard] =
+  var p = initVCardParser(input, filename)
+  while p.peek != '\0': result.add(p.readVCard)
+
+proc parseVCards*(content: string, filename = "input"): seq[VCard] =
+  parseVCards(newStringStream(content), filename)
+
+proc parseVCardsFromFile*(filepath: string): seq[VCard] =
+  parseVCards(newFileStream(filepath, fmRead), filepath)
--- a/src/vcard/common.nim
+++ b/src/vcard/common.nim
@ -0,0 +1,91 @@
+# Common Functionality
+# © 2022-2023 Jonathan Bernard
+
+## This module contains type definitions and func/proc definitions that are
+## used in common between both the 3.0 and 4.0 parser implementations.
+import std/[options, sequtils]
+import zero_functional
+import ./private/lexer
+
+type
+  VC_Param* = tuple[name: string, values: seq[string]]
+    ## Representation of vCard parameter and its values.
+
+  VCardVersion* = enum
+    ## enum used to differentiate VCard3 and VCard4 versions.
+    VCardV3 = "3.0", VCardV4 = "4.0"
+
+  VCardParser* = object of VCardLexer
+    ## Common vCard parser object
+    filename*: string
+
+  VCardParsingError* = object of ValueError
+    ## Error raised when invalid input is detected while parsing a vCard
+
+  VC_XParam* = tuple[name, value: string]
+    ## Representation of vCard extended parameters (starting with "X-").
+    ## Because the meaning of these parameters is implementation-specific, no
+    ## parsing of the parameter value is performed, it is returned verbatim.
+
+  VCard* = ref object of RootObj
+    ## Abstract base class for all vCards. `parsedVersion` can be used to
+    ## interrogate any concrete instance of this class. `asVCard3` and
+    ## `asVCard4` exist as convenience functions to cast an instance to one of
+    ## the subclasses depending on the value of `parsedVersion`.
+    parsedVersion*: VCardVersion
+
+proc getMultipleValues*(
+    params: openarray[VC_Param],
+    name: string
+  ): seq[string] =
+
+  ## Get all of the values for a given parameter in a single list. There are
+  ## two patterns for multi-valued parameters defined in the vCard 3.0 RFCs:
+  ##
+  ##   - TYPE=work,cell,voice
+  ##   - TYPE=work;TYPE=cell;TYPE=voice
+  ##
+  ## Parameter values can be specific using both patterns. This method joins
+  ## all defined values regardless of the pattern used to define them.
+
+  let ps = params.toSeq
+  ps -->
+    filter(it.name == name).
+    map(it.values).
+    flatten()
+
+proc getSingleValue*(
+    params: openarray[VC_Param],
+    name: string
+  ): Option[string] =
+  ## Get the first single value defined for a parameter.
+  ##
+  ## Many parameters only support a single value, depending on the content type.
+  ## In order to support multi-valued parameters our implementation stores all
+  ## parameters as seq[string]. This function is a convenience around that.
+
+  let ps = params.toSeq
+  let foundParam = ps --> find(it.name == name)
+
+  if foundParam.isSome and foundParam.get.values.len > 0:
+    return some(foundParam.get.values[0])
+  else:
+    return none[string]()
+
+func allPropsOfType*[T, VC: VCard](vc: VC): seq[T] =
+  ## Get all instances of the requested property type present on the given
+  ## vCard.
+  ##
+  ## This can be useful when there is some logic that hides multiple instances
+  ## of a property, or returns a limited subset. For example, on 3.0 versions
+  ## of vCards, this library assumes that there will only be one instance of
+  ## the NAME property. The 3.0 spec implies that the NAME property should only
+  ## be present at most once, but does not explicitly state this. It is
+  ## possible for a 3.0 vCard to contain multiple NAME properties. using
+  ## `vc3.name` will only return the first. This function allows a caller to
+  ## retrieve all instances for any given property type. For example:
+  ##
+  ## .. code-block:: nim
+  ##   let vc3 = parseVCards(...)
+  ##   let allNames = allPropsOfType[VC3_Name](vc3)
+  vc.content.filterIt(it of typeof(T)).mapIt(cast[T](it))
--- a/src/vcard/private/internals.nim
+++ b/src/vcard/private/internals.nim
@ -0,0 +1,283 @@
+# Common Functionality
+# © 2022-2023 Jonathan Bernard
+
+## This module contains type definitions, constant values, and func/proc
+## definitions that are used in common between both the 3.0 and 4.0 parser
+## implementations.
+##
+## This module is not intended to be exposed to library consumers. It is
+## intended to be imported by the vcard3 and vcard4 implementations.
+
+import std/[macros, options, strutils, times, unicode]
+import zero_functional
+from std/sequtils import toSeq
+
+import ./lexer
+import ../common
+
+# Internal Types (used by `vcard{3,4}`)
+# =====================================
+type
+  VC_PropCardinality* = enum
+    ## enum used to define the possible cardinalities of vCard properties.
+    vpcAtMostOne,
+    vpcExactlyOne,
+    vpcAtLeastOne
+    vpcAny
+
+# Internal constants (used by `vcard{3,4}`)
+# =========================================
+const CRLF* = "\r\n"
+const WSP* = {' ', '\t'}
+const DIGIT* = { '0'..'9' }
+const ALPHA_NUM* = { 'a'..'z', 'A'..'Z', '0'..'9' }
+const NON_ASCII* = { '\x80'..'\xFF' }
+const QSAFE_CHARS* = WSP + { '\x21', '\x23'..'\x7E' } + NON_ASCII
+const SAFE_CHARS* = WSP + { '\x21', '\x23'..'\x2B', '\x2D'..'\x39', '\x3C'..'\x7E' } + NON_ASCII
+const VALUE_CHAR* = WSP + { '\x21'..'\x7E' } + NON_ASCII
+
+const DATE_FMTS = [ "yyyy-MM-dd", "yyyyMMdd" ]
+const DATE_TIME_FMTS = [
+  "yyyyMMdd'T'HHmmss",
+  "yyyyMMdd'T'HHmmssz",
+  "yyyyMMdd'T'HHmmsszzz",
+  "yyyyMMdd'T'HHmmss'.'fffzzz",
+  "yyyy-MM-dd'T'HH:mm:ss",
+  "yyyy-MM-dd'T'HH:mm:ssz",
+  "yyyy-MM-dd'T'HH:mm:sszzz",
+  "yyyy-MM-dd'T'HH:mm:ss'.'fffzzz",
+]
+
+const ALL_DATE_AND_OR_TIME_FMTS = DATE_TIME_FMTS.toSeq & DATE_FMTS.toSeq
+
+# Internal Utility/Implementation Functions
+# =========================================
+
+proc parseDateTimeStr(
+    dateStr: string,
+    dateFmts: openarray[string]
+  ): DateTime {.inline, raises:[ValueError].} =
+
+  for fmt in dateFmts:
+    try: result = parse(dateStr, fmt)
+    except ValueError: discard
+
+  if not result.isInitialized:
+    raise newException(ValueError, "cannot parse date: " & dateStr )
+
+proc parseDate*(dateStr: string): DateTime =
+  parseDateTimeStr(dateStr, DATE_FMTS)
+
+proc parseDateTime*(dateStr: string): DateTime =
+  parseDateTimeStr(dateStr, DATE_TIME_FMTS)
+
+proc parseDateOrDateTime*(dateStr: string): DateTime =
+  parseDateTimeStr(dateStr, ALL_DATE_AND_OR_TIME_FMTS)
+
+template indexOfIt*(s, pred: untyped): int =
+  var result = -1
+  for idx, it {.inject.} in pairs(s):
+    if pred:
+      result = idx
+      break
+  result
+
+template findAll*[T, VCT](c: openarray[VCT]): seq[T] =
+  c.filterIt(it of typeof(T)).mapIt(cast[T](it))
+
+template findFirst*[T, VCT](c: openarray[VCT]): Option[T] =
+  let found = c.filterIt(it of typeof(T)).mapIt(cast[T](it))
+  if found.len > 0: some(found[0])
+  else: none[T]()
+
+macro assignFields*(assign: untyped, fields: varargs[untyped]): untyped =
+  result = assign
+
+  for f in fields:
+    let exp = newNimNode(nnkExprColonExpr)
+    exp.add(f, f)
+    result.add(exp)
+
+# Internal Parsing Functionality
+# ==============================
+
+proc error*(p: VCardParser, msg: string) =
+  raise newException(VCardParsingError, "$1($2, $3) Error: $4" %
+    [ p.filename, $p.lineNumber, $p.getColNumber(p.pos), msg ])
+
+proc expect*(p: var VCardParser, expected: string, caseSensitive = false) =
+  ## Read `expected.len` from the parser's input stream and determine if it
+  ## matches the expected value. If the parser is unable to supply
+  ## `expected.len` bytes or if the value read does not match the expected
+  ## value, raise a VCardParsingError via the `error` utility function
+  ## reporting to the user where the input failed to match the expected value.
+  ## If the expectation is met, the parser read position is advanced past the
+  ## expected value, on the next byte after the expected value that was read.
+  try:
+    p.setBookmark
+
+    if caseSensitive:
+      for ch in expected:
+        if p.read != ch: raise newException(ValueError, "")
+    else:
+      for rune in expected.runes:
+        if p.readRune.toLower != rune.toLower:
+          raise newException(ValueError, "")
+
+  except ValueError:
+    p.error("expected '$1' but found '$2':\n\t$3\n\t$4" %
+      [expected, p.readSinceBookmark, p.lineVal,
+       " ".repeat(p.getColNumber(p.pos) - 1) & "^\n"])
+
+  finally: p.unsetBookmark
+
+proc isNext*(p: var VCardParser, expected: string, caseSensitive = false): bool =
+  ## Read `expected.len` from the parser's input stream and determine if it
+  ## matches the expected value. Assuming the parser did not fail to read
+  ## `expected.len` bytes, this will reset the parser read state to its initial
+  ## position (prior to calling `isNext`). This is similar to `expect` but
+  ## exhibits more "`peek`-like" behavior (doesn't disturb the read position,
+  ## and doesn't fail).
+  result = true
+  p.setBookmark
+
+  if caseSensitive:
+    for ch in expected:
+      if p.read != ch:
+        result = false
+        break
+
+  else:
+    for rune in expected.runes:
+      if p.readRune.toLower != rune.toLower:
+        result = false
+        break
+
+  p.returnToBookmark
+
+proc readGroup*(p: var VCardParser): Option[string] =
+  ## All vCard content items can be optionally prefixed with a group name. This
+  ## scans the input to see if there is a group defined at the current read
+  ## location. If there is a valid group, the group name is returned and the
+  ## read position is advanced past the '.' to the start of the content type
+  ## name. If there is not a valid group the read position is left unchanged.
+
+  p.setBookmark
+  var ch = p.read
+  while ALPHA_NUM.contains(ch): ch = p.read
+
+  if (ch == '.'):
+    result = some(readSinceBookmark(p)[0..^2])
+    p.unsetBookmark
+  else:
+    result = none[string]()
+    p.returnToBookmark
+
+proc readName*(p: var VCardParser): string =
+  ## Read a name from the current read position or error. As both content types
+  ## and paramaters use the same definition for valid names, this method is
+  ## used to read in both.
+  p.setBookmark
+  let validChars = ALPHA_NUM + {'-'}
+  while validChars.contains(p.peek): discard p.read
+  result = p.readSinceBookmark.toUpper()
+  if result.len == 0:
+    p.error("expected to read a name but found '$1'" % [$p.peek])
+  p.unsetBookmark
+
+proc readValue*(p: var VCardParser): string =
+  ## Read a content value at the current read position.
+  p.setBookmark
+  while VALUE_CHAR.contains(p.peek): discard p.read
+  result = p.readSinceBookmark
+  p.unsetBookmark
+
+proc skip*(p: var VCardParser, count: int): bool =
+  ## Ignore the next `count` bytes of data from the parser's underlying input
+  ## stream.
+  for _ in 0..<count: discard p.read
+
+proc skip*(p: var VCardParser, expected: string, caseSensitive = false): bool =
+
+  ## Ignore the next `expected.len` bytes of data from the parser's underlying
+  ## input stream, but only if the value read matches the value provided in
+  ## `expected`. In other words: read `expected.len` bytes. If they match the
+  ## expectation, leave the parser read position in the new state. If they do
+  ## not match, reset the parser read position to the state prior to invoking
+  ## `skip`.
+
+  p.setBookmark
+  if caseSensitive:
+    for ch in expected:
+      if p.read != ch:
+        p.returnToBookmark
+        return false
+
+  else:
+    for rune in expected.runes:
+      if p.readRune.toLower != rune.toLower:
+        p.returnToBookmark
+        return false
+
+  p.unsetBookmark
+  return true
+
+proc existsWithValue*(
+    params: openarray[VC_Param],
+    name, value: string,
+    caseSensitive = false
+  ): bool =
+
+  ## Determine if the given parameter exists and has the expected value. By
+  ## default, value checks are not case-sensitive, as most VCard3 values are not
+  ## defined as being case-sensitive.
+
+  let ps = params.toSeq
+
+  if caseSensitive:
+    ps --> exists(
+      it.name == name and
+      it.values.len == 1 and
+      it.values[0] == value)
+  else:
+    ps --> exists(
+      it.name == name and
+      it.values.len == 1 and
+      it.values[0].toLower == value.toLower)
+
+proc validateNoParameters*(
+    p: VCardParser,
+    params: openarray[VC_Param],
+    name: string
+  ) =
+
+  ## Raise a VCardParsingError if there are any parameters.
+  if params.len > 0:
+    p.error("no parameters allowed on the $1 content type" % [name])
+
+proc validateRequiredParameters*(
+    p: VCardParser,
+    params: openarray[VC_Param],
+    expectations: openarray[tuple[name: string, value: string]]
+  ) =
+
+  ## Some content types have specific allowed parameters. For example, the
+  ## SOURCE content type requires that the VALUE parameter be set to "uri" if
+  ## it is present. This will raise a VCardParsingError if given parameters are
+  ## present with different values that expected.
+
+  for (n, v) in expectations:
+    let pv = params.getSingleValue(n)
+    if pv.isSome and pv.get != v:
+      p.error("parameter '$1' must have the value '$2'" % [n, v])
+
+# Internal Serialization Utilities
+# ================================
+
+func foldContentLine*(s: string): string =
+  result = ""
+  var rem = s
+  while rem.len > 75: # TODO: unicode multi-byte safe?
+    result &= rem[0..<75] & "\r\n "
+    rem = rem[75..^1]
+  result &= rem
--- a/src/vcard/private/lexer.nim
+++ b/src/vcard/private/lexer.nim
@ -1,25 +1,45 @@
+# vCard-specific Lexer
+# © 2022-2023 Jonathan Bernard
+
+## This module defines a lexer with functionality useful for parsing vCard
+## content. Specifically:
+## - it understands the vCard line-folding logic and transparently joins folded
+##   lines as it read input off its input stream.
+## - it supports multiple nested bookmarks to make look-ahead decisions more
+##   convenient
+## - it supports reading from the underlying stream byte-wise or as unicode
+##   runes.
+##
+## This parser uses a ring buffer underneath, only growing the size of its
+## buffer when it is completely full.
+
 import std/[streams, unicode]

 type VCardLexer* = object of RootObj
  input: Stream

-  buffer*: string       # buffer of bytes read
-  bufStart: int         # starting boundary for the buffer
-  bufEnd: int           # ending boundary for the buffer
-  pos*: int             # current read position
-  bookmark*: int        # bookmark to support rewind functionality
-  bookmarkVal*: string  # value that has been read since the bookmark was set
-  lineNumber*: int      # how many newlines have we seen so far
-  lineStart: int        # index into the buffer for the start of the current line
+  buffer*: string           ## buffer of bytes read
+  bufStart: int             ## starting boundary for the buffer
+  bufEnd: int               ## ending boundary for the buffer
+  pos*: int                 ## current read position
+  bookmark*: seq[int]       ## bookmark to support rewind functionality
+  bookmarkVal*: seq[string] ## value read since the bookmark was set
+  lineNumber*: int          ## how many newlines have we seen so far
+  lineStart: int            ## buffer index buffer for the start of the current line
+  lineVal*: string          ## value read since the start of the current line

 proc skipUtf8Bom(vcl: var VCardLexer) =
  if (vcl.buffer[0] == '\xEF') and (vcl.buffer[1] == '\xBB') and (vcl.buffer[2] == '\xBF'):
    inc(vcl.pos, 3)

 template wrappedIdx(idx: untyped): int = idx mod vcl.buffer.len
+  ## Map an index into the buffer bounds (mod)

 proc newStartIdx(vcl: VCardLexer): int =
-  if vcl.bookmark > 0: vcl.bookmark else: vcl.pos
+  ## Get the latest safe index to use as a new start index. The implication is
+  ## that anything prior to this index has been read and processed and can be
+  ## safely overwritten in the buffer.
+  if vcl.bookmark.len > 0: vcl.bookmark[0] else: vcl.pos

 func isFull(vcl: VCardLexer): bool {.inline.} =
  return wrappedIdx(vcl.bufEnd + 1) == vcl.newStartIdx
@ -28,6 +48,8 @@ func atEnd(vcl: VCardLexer): bool {.inline.} =
  vcl.pos == vcl.bufEnd

 proc doubleBuffer(vcl: var VCardLexer) =
+  ## Double the capacity of the buffer, copying the contents of the current
+  ## buffer into the beginning of the newly expanded buffer.
  let oldBuf = vcl.buffer
  vcl.buffer = newString(oldBuf.len * 2)

@ -39,13 +61,28 @@ proc doubleBuffer(vcl: var VCardLexer) =
    inc(newIdx)
    oldIdx = (newIdx + vcl.bufStart) mod oldBuf.len

+  # We know that for all existing indices, their location in the new buffer can
+  # be calculated as a function of the distance we moved the start of the
+  # buffer: `idx = idx + (newBufStart - oldBufStart)`. Since we know the new
+  # bufStart will be 0, we know that we can calculate all of the new indices as
+  # `idx -= oldBufStart` Currently vcl.bufStart is still the old bufStart.
  vcl.pos -= vcl.bufStart
  vcl.lineStart -= vcl.bufStart
-  if vcl.bookmark >= 0: vcl.bookmark -= vcl.bufStart
+  if vcl.bookmark.len > 0: vcl.bookmark[0] -= vcl.bufStart
+
+  # Now that we've updated all of the existing indices, we can reset the
+  # buffer start and end indices to their new values.
  vcl.bufStart = 0
  vcl.bufEnd = newIdx

 proc fillBuffer(vcl: var VCardLexer) =
+  ## Read data into the buffer from the underlying stream until the buffer is
+  ## full. If the buffer is already full, double the buffer beforehand.
+
+  # Note that we do not *completely* fill the buffer. We always leave one index
+  # of the array empty. This allows us to differentiate between an empty buffer
+  # (`bufStart == bufEnd`) and a completly full buffer (`bufStart ==
+  # wrappedIdx(bufEnd + 1)`).

  var charsRead: int

@ -55,7 +92,10 @@ proc fillBuffer(vcl: var VCardLexer) =
  # discard used portions of the buffer
  vcl.bufStart = vcl.newStartIdx

+  # We have three conditions that the ring buffer may be in:
  if vcl.bufEnd < vcl.bufStart:
+    # The unused portion of the buffer is all in the middle and we can just
+    # read date into the space from bufEnd (e) to bufStart (s).
    #     e       s
    # 0 1 2 3 4 5 6 7 8 9
    charsRead = vcl.input.readDataStr(vcl.buffer,
@ -63,6 +103,8 @@ proc fillBuffer(vcl: var VCardLexer) =
    vcl.bufEnd += charsRead

  elif vcl.bufStart == 0:
+    # The unused portion is entirely at the end of the buffer. We can read data
+    # from the bufEnd (e) to the end of our buffer capacity.
    # s           e
    # 0 1 2 3 4 5 6 7 8 9
    charsRead = vcl.input.readDataStr(vcl.buffer,
@ -70,21 +112,29 @@ proc fillBuffer(vcl: var VCardLexer) =
    vcl.bufEnd = wrappedIdx(vcl.bufEnd + charsRead)

  else:
+    # The used portion of the buffer is in the middle, and the unused portion
+    # is on either side ot that. We need to read from bufEnd (e) to the end of
+    # our underlying buffer, and then from the start of our underlying buffer
+    # to bufStart (s)
    #     s       e
    # 0 1 2 3 4 5 6 7 8 9
    charsRead = vcl.input.readDataStr(vcl.buffer, vcl.bufEnd..<vcl.buffer.len)
    if charsRead == vcl.buffer.len - vcl.bufEnd:
+      # Only read into the front part of the buffer if we were able to
+      # completely fill the back part.
      vcl.bufEnd = vcl.input.readDataStr(vcl.buffer, 0 ..< (vcl.bufStart - 1))


 proc close*(vcl: var VCardLexer) = vcl.input.close
+  ## Close this VCardLexer and its underlying stream.

 proc open*(vcl: var VCardLexer, input: Stream, bufLen = 16384) =
+  ## Open the given stream and initialize the given VCardLexer to read from it.
  assert(bufLen > 0)
  assert(input != nil)
  vcl.input = input
  vcl.pos = 0
-  vcl.bookmark = -1
+  vcl.bookmark = @[]
  vcl.buffer = newString(bufLen)
  vcl.bufStart = 0
  vcl.bufEnd = 0
@ -94,38 +144,56 @@ proc open*(vcl: var VCardLexer, input: Stream, bufLen = 16384) =
  vcl.skipUtf8Bom

 proc setBookmark*(vcl: var VCardLexer) =
-  vcl.bookmark = vcl.pos
-  vcl.bookmarkVal = newStringOfCap(32)
+  ## Set a bookmark into the lexer's buffer. This will prevent the lexer from
+  ## discarding data from this bookmark forward when it refills.
+  ##
+  ## The bookmark must be cleared using either `returnToBookmark` or
+  ## `unsetBookmark`, otherwise this lexer will no function as a streaming
+  ## lexer and will end up reading the entire remainder of the input stream
+  ## into memory (if it can).
+  ##
+  ## This function can be called multiple times in order to create nested
+  ## bookmarks. For example, we might set a bookmark at the beginning of a line
+  ## to be able to reset if we fail to parse the line, then set a bookmark
+  ## midway through when attempting to parse a parameter value. Care should be
+  ## taken when nesting bookmarks as all bookmarks must be released to avoid
+  ## the behavior described above.
+  vcl.bookmark.add(vcl.pos)
+  vcl.bookmarkVal.add(newStringOfCap(32))

 proc returnToBookmark*(vcl: var VCardLexer) =
-  vcl.pos = vcl.bookmark
-  vcl.bookmark = -1
+  ## Unset the most recent bookmark, resetting the lexer's read position to the
+  ## position of the bookmark.
+  if vcl.bookmark.len == 0: return
+  vcl.pos = vcl.bookmark.pop()
+  let valRead = vcl.bookmarkVal.pop()
+  for idx in 0..<vcl.bookmarkVal.len:
+    if vcl.bookmarkVal[idx].len > valRead.len:
+      vcl.bookmarkVal[idx] = vcl.bookmarkVal[idx][0 ..< ^valRead.len]

 proc unsetBookmark*(vcl: var VCardLexer) =
-  vcl.bookmark = -1
+  ## Discard the most recent bookmark, leaving the lexer's read position at its
+  ## current position..
+  if vcl.bookmark.len == 0: return
+  discard vcl.bookmark.pop()
+  discard vcl.bookmarkVal.pop()

 proc readSinceBookmark*(vcl: var VCardLexer): string =
-  return vcl.bookmarkVal
-#[
-  if vcl.pos < vcl.bookmark:
-    #     p e   s b
-    # 0 1 2 3 4 5 6 7 8 9
-    result = newStringOfCap(vcl.buffer.len - vcl.bookmark + vcl.pos)
-  else:
-    #   s   b     p   e
-    # 0 1 2 3 4 5 6 7 8 9
-    result = newStringOfCap(vcl.pos - vcl.bookmark)
-
-  let curPos = vcl.pos
-  vcl.pos = vcl.bookmark
-  while vcl.pos != curPos: result.add(vcl.read)
-]#
+  ## Get the value read since the last bookmark.
+  if vcl.bookmarkVal.len > 0:
+    return vcl.bookmarkVal[^1]
+  else: return ""

 proc isLineWrap(vcl: var VCardLexer, allowRefill = true): bool =
+  ## Answers the question "is this a folded line"? Transparently handles the
+  ## case where it needs to refill the buffer to answer this question.
  if vcl.buffer[vcl.pos] != '\r': return false

  # less than three characters in the buffer
  if wrappedIdx(vcl.pos + 3) > vcl.bufEnd:
+    # Only try to refill the buffer once. If we re-enter and still don't have
+    # three characters, we know we were unable to fill the buffer and are
+    # likely at the end.
    if allowRefill:
      vcl.fillBuffer()
      return vcl.isLineWrap(false)
@ -137,54 +205,87 @@ proc isLineWrap(vcl: var VCardLexer, allowRefill = true): bool =
           vcl.buffer[wrappedIdx(vcl.pos + 2)] == ' '

 proc read*(vcl: var VCardLexer, peek = false): char =
+  ## Read one byte off of the input stream. By default this will advance the
+  ## lexer read position by one byte. If `peek` is set to `true`, this will
+  ## leave the read position at the same logical position. The underlying
+  ## buffer position may still change if, for example, the next byte is the
+  ## beginning of a folded line wrap. In this case the internal buffer position
+  ## will advance past that line wrap.
+
  if vcl.atEnd: vcl.fillBuffer()

  if vcl.isLineWrap:
    vcl.pos += 3
    vcl.lineNumber += 1
    vcl.lineStart = vcl.pos
+    vcl.lineVal = newStringOfCap(84)
    if vcl.atEnd: vcl.fillBuffer()

-  elif vcl.buffer[vcl.pos] == '\n':
+  elif vcl.buffer[vcl.pos] == '\n' and not peek:
    vcl.lineNumber += 1
    vcl.lineStart = wrappedIdx(vcl.pos + 1)
+    vcl.lineVal = newStringOfCap(84)

  result = vcl.buffer[vcl.pos]
  if not peek:
-    if vcl.bookmark != -1: vcl.bookmarkVal.add(result)
+    for idx in 0..<vcl.bookmarkVal.len: vcl.bookmarkVal[idx].add(result)
+    vcl.lineVal.add(result)
    vcl.pos = wrappedIdx(vcl.pos + 1)

+proc readLen*(vcl: var VCardLexer, bytesToRead: int, peek = false): string =
+  ## Convenience procedure to read multiple bytes (if able) and return the
+  ## value read.
+  result = newStringOfCap(bytesToRead)
+  for i in 0..<bytesToRead: result.add(vcl.read)
+
 proc readRune*(vcl: var VCardLexer, peek = false): Rune =
+  ## Read one unicode rune off of the input stream. By default this will
+  ## advance the lexer read position by the byte length of the rune read. If
+  ## `peek` is set to `true`, this will leave the read position at the same
+  ## logical position. The underlying buffer position may still change if, for
+  ## example, the next rune is the beginning of a folded line wrap. In this
+  ## case the internal buffer position will advance past that line wrap.
+
  if vcl.atEnd: vcl.fillBuffer()

  if vcl.isLineWrap:
    vcl.pos += 3
    vcl.lineNumber += 1
    vcl.lineStart = vcl.pos
+    vcl.lineVal = newStringOfCap(84)
    if vcl.atEnd: vcl.fillBuffer()

  elif vcl.buffer[vcl.pos] == '\n':
    vcl.lineNumber += 1
    vcl.lineStart = wrappedIdx(vcl.pos + 1)
+    vcl.lineVal = newStringOfCap(84)

  result = vcl.buffer.runeAt(vcl.pos)
-  if not peek: vcl.pos += vcl.buffer.runeLenAt(vcl.pos)
+  if not peek:
+    for idx in 0..<vcl.bookmarkVal.len: vcl.bookmarkVal[idx].add(result)
+    vcl.lineVal.add(result)
+    vcl.pos += vcl.buffer.runeLenAt(vcl.pos)
+
+proc readRunesLen*(vcl: var VCardLexer, runesToRead: int, peek = false): string =
+  ## Convenience procedure to read multiple runes (if able) and return the
+  ## value read.
+  result = newStringOfCap(runesToRead * 4)
+  for i in 0..<runesToRead: result.add(vcl.readRune)

 proc peek*(vcl: var VCardLexer): char =
+  ## Convenience method to call `read(peek = true)`
  return vcl.read(peek = true)

 proc peekRune*(vcl: var VCardLexer): Rune =
+  ## Convenience method to call `read(peek = true)`
  return vcl.readRune(peek = true)

 proc getColNumber*(vcl: VCardLexer, pos: int): int =
+  ## Calculate the column number of the lexer's current read position relative
+  ## to the start of the most recent line.
  if vcl.lineStart < pos: return pos - vcl.lineStart
  else: return (vcl.buffer.len - vcl.lineStart) + pos

-## Unit Tests
-## ============================================================================
-
-import std/unittest
-
 proc dumpLexerState*(l: VCardLexer): string =
  result =
    "pos        = " & $l.pos & "\p" &
@ -195,7 +296,9 @@ proc dumpLexerState*(l: VCardLexer): string =
    "bufEnd     = " & $l.bufEnd & "\p" &
    "buffer     = " & l.buffer & "\p"

-suite "vcard/lexer":
+## Unit Tests
+## ============================================================================
+proc runVcardLexerPrivateTests*() =

  const longTestString =
    "This is my test string. There are many like it but this one is mine."
@ -212,143 +315,204 @@ suite "vcard/lexer":
        return false
    return true

-  #test "fillBuffer doesn't double the buffer needlessly":
-  #  var l: VCardLexer
-
  proc readExpected(vcl: var VCardLexer, s: string): bool =
    for i in 0..<s.len:
      if vcl.read != s[i]:
        return false
    return true

-  test "can open and fill buffer":
+  # "can open and fill buffer":
+  block:
    var l: VCardLexer
    l.open(newStringStream("test"))
-    check:
-      l.bufferIs("test")
-      not l.isFull
-      l.readExpected("test")
+    assert l.bufferIs("test")
+    assert not l.isFull
+    assert l.readExpected("test")

-  test "refills buffer when emptied":
+  # "refills buffer when emptied":
+  block:
    var l: VCardLexer
    l.open(newStringStream("test"), 3)
-    check:
-      l.bufferIs("te")
-      l.isFull
-      l.read == 't'
-      l.read == 'e'
-      l.read == 's'
-      l.bufferIs("st")
-      l.read == 't'
+    assert l.bufferIs("te")
+    assert l.isFull
+    assert l.read == 't'
+    assert l.read == 'e'
+    assert l.read == 's'
+    assert l.bufferIs("st")
+    assert l.read == 't'

-  test "isFull correctness":
+  # "isFull correctness":
+  block:
    var l = VCardLexer(
      pos: 0,
-      bookmark: -1,
+      bookmark: @[],
      buffer: "0123456789",
      bufStart: 0,
      bufEnd: 9)

    # s                 e
    # 0 1 2 3 4 5 6 7 8 9
-    check l.isFull
+    assert l.isFull

    # s p               e
    # 0 1 2 3 4 5 6 7 8 9
    discard l.read
-    check not l.isFull
+    assert not l.isFull

    #     e s
    # 0 1 2 3 4 5 6 7 8 9
    l.bufStart = 3
    l.pos = 3
    l.bufEnd = 2
-    check l.isFull
+    assert l.isFull

    #     e s p
    # 0 1 2 3 4 5 6 7 8 9
    discard l.read
-    check:
-      l.pos == 4
-      not l.isFull
+    assert l.pos == 4
+    assert not l.isFull

    #                 e s
    # 0 1 2 3 4 5 6 7 8 9
    l.bufStart = 9
    l.pos = 9
    l.bufEnd = 8
-    check l.isFull
+    assert l.isFull

    # p               e s
    # 0 1 2 3 4 5 6 7 8 9
    discard l.read
-    check:
-      l.pos == 0
-      not l.isFull
+    assert l.pos == 0
+    assert not l.isFull

-  test "handles wrapped lines":
+  # "handles wrapped lines":
+  block:
    var l: VCardLexer
    l.open(newStringStream("line\r\n  wrap\r\nline 2"), 3)

-    check l.readExpected("line wrap\r\nline 2")
+    assert l.readExpected("line wrap\r\nline 2")

-  test "fillBuffer correctness":
+  # "fillBuffer correctness":
+  block:
    var l: VCardLexer
    l.open(newStringStream(longTestString), 5)
-    check:
-      l.bufferIs(longTestString[0..<4])
-      l.isFull
-      l.bufStart == 0
-      l.bufEnd == 4
-      l.pos == 0
-      l.readExpected("Th")
-      not l.isFull
-      not l.atEnd
-      l.pos == 2
+    assert l.bufferIs(longTestString[0..<4])
+    assert l.isFull
+    assert l.bufStart == 0
+    assert l.bufEnd == 4
+    assert l.pos == 0
+    assert l.readExpected("Th")
+    assert not l.isFull
+    assert not l.atEnd
+    assert l.pos == 2

    l.fillBuffer
-    check:
-      l.isFull
-      l.bufEnd == 1
-      l.pos == 2
-      l.bufStart == 2
+    assert l.isFull
+    assert l.bufEnd == 1
+    assert l.pos == 2
+    assert l.bufStart == 2

-  test "bookmark preserves the buffer":
+  # "bookmark preserves the buffer":
+  block:
    var l: VCardLexer
    l.open(newStringStream(longTestString), 7)
-    check:
-      l.buffer.len == 7
-      l.bufferIs(longTestString[0..<6])
-      l.isFull
-      l.bufEnd == 6
-      l.pos == 0
-      l.bookmark == -1
-      l.readExpected(longTestString[0..<5])
-      not l.isFull
-      not l.atEnd
-      l.pos == 5
+    assert l.buffer.len == 7
+    assert l.bufferIs(longTestString[0..<6])
+    assert l.isFull
+    assert l.bufEnd == 6
+    assert l.pos == 0
+    assert l.bookmark == @[]
+    assert l.readExpected(longTestString[0..<5])
+    assert not l.isFull
+    assert not l.atEnd
+    assert l.pos == 5

    l.setBookmark
    # read enough to require us to refill the buffer.
-    check:
-      l.bookmark == 5
-      l.readExpected(longTestString[5..<10])
-      l.pos == 3
-      newStartIdx(l) == 5
-      l.buffer.len == 7
+    assert l.bookmark == @[5]
+    assert l.readExpected(longTestString[5..<10])
+    assert l.pos == 3
+    assert newStartIdx(l) == 5
+    assert l.buffer.len == 7

    l.returnToBookmark
-    check:
-      l.bookmark == -1
-      l.pos == 5
+    assert l.bookmark == @[]
+    assert l.pos == 5

-  test "readRune":
+  # "can set and unset multiple bookmarks"
+  block:
+    var l: VCardLexer
+    l.open(newStringStream(longTestString))
+    assert l.pos == 0
+    assert l.bookmark == @[]
+    assert l.readExpected("This is my ")
+
+    l.setBookmark
+    assert l.bookmark == @[11]
+    assert l.bookmarkVal == @[""]
+
+    assert l.readExpected("test string")
+    assert l.bookmark == @[11]
+    assert l.bookmarkVal == @["test string"]
+    assert l.readSinceBookmark == "test string"
+
+    l.setBookmark
+    assert l.bookmark == @[11, 22]
+    assert l.bookmarkVal == @["test string", ""]
+
+    assert l.readExpected(". There are many")
+    assert l.bookmarkVal == @["test string. There are many", ". There are many"]
+    assert l.readSinceBookmark == ". There are many"
+    assert l.pos == 38
+
+    l.unsetBookmark
+    assert l.pos == 38
+    assert l.bookmark == @[11]
+    assert l.bookmarkVal == @["test string. There are many"]
+    assert l.readSinceBookmark == "test string. There are many"
+
+    l.unsetBookmark
+    assert l.pos == 38
+    assert l.bookmark == @[]
+    assert l.bookmarkVal == @[]
+    assert l.readSinceBookmark == ""
+
+  # "can set and return to multiple bookmarks"
+  block:
+    var l: VCardLexer
+    l.open(newStringStream(longTestString))
+    assert l.pos == 0
+    assert l.bookmark == @[]
+    assert l.readExpected("This is my ")
+
+    l.setBookmark
+    assert l.readExpected("test string")
+    l.setBookmark
+    assert l.bookmark == @[11, 22]
+    assert l.readExpected(". There are many")
+    assert l.bookmarkVal == @["test string. There are many", ". There are many"]
+    assert l.pos == 38
+
+    l.returnToBookmark
+    assert l.pos == 22
+    assert l.bookmark == @[11]
+    assert l.bookmarkVal == @["test string"]
+    assert l.readSinceBookmark == "test string"
+
+    l.returnToBookmark
+    assert l.pos == 11
+    assert l.bookmark == @[]
+    assert l.bookmarkVal == @[]
+
+  # "readRune":
+  block:
    var l: VCardLexer
    l.open(newStringStream("TEST"))
-    check:
-      l.bufferIs("TEST")
-      l.peekRune == Rune('T')
-      l.readRune == Rune('T')
-      l.readRune == Rune('E')
-      l.readRune == Rune('S')
-      l.readRune == Rune('T')
+    assert l.bufferIs("TEST")
+    assert l.peekRune == Rune('T')
+    assert l.readRune == Rune('T')
+    assert l.readRune == Rune('E')
+    assert l.readRune == Rune('S')
+    assert l.readRune == Rune('T')
+
+when isMainModule: runVcardLexerPrivateTests()
--- a/src/vcard/private/parsercommon.nim
+++ b/src/vcard/private/parsercommon.nim
@ -1,40 +0,0 @@
-import options, strutils
-import ./lexer
-
-const WSP* = {' ', '\t'}
-const ALPHA_NUM* = { 'a'..'z', 'A'..'Z', '0'..'9' }
-
-proc expect*[T](p: var T, expected: string, caseSensitive = false) =
-  p.setBookmark
-
-  if caseSensitive:
-    for ch in expected:
-      if p.read != ch:
-        p.error("expected '$1' but found '$2'" %
-          [expected, p.readSinceBookmark])
-
-  else:
-    for rune in expected.runes:
-      if p.readRune.toLower != rune.toLower:
-        p.error("expected '$1' but found '$2'" %
-          [ expected, p.readSinceBookmark ])
-
-  p.unsetBookmark
-
-proc readGroup*[T](p: var T): Option[string] =
-  ## All VCARD content items can be optionally prefixed with a group name. This
-  ## scans the input to see if there is a group defined at the current read
-  ## location. If there is a valid group, the group name is returned and the
-  ## read position is advanced past the '.' to the start of the content type
-  ## name. If there is not a valid group the read position is left unchanged.
-
-  p.setBookmark
-  var ch = p.read
-  while ALPHA_NUM.contains(ch): ch = p.read
-
-  if (ch == '.'):
-    result = some(readSinceBookmark(p)[0..^2])
-    p.unsetBookmark
-  else:
-    result = none[string]()
-    p.returnToBookmark
--- a/src/vcard/private/util.nim
+++ b/src/vcard/private/util.nim
@ -1,55 +0,0 @@
-import sequtils, strutils, times
-
-const DATE_FMTS = [ "yyyy-MM-dd", "yyyyMMdd" ]
-const DATE_TIME_FMTS = [
-  "yyyyMMdd'T'HHmmss",
-  "yyyyMMdd'T'HHmmssz",
-  "yyyyMMdd'T'HHmmsszzz",
-  "yyyyMMdd'T'HHmmss'.'fffzzz",
-  "yyyy-MM-dd'T'HH:mm:ss",
-  "yyyy-MM-dd'T'HH:mm:ssz",
-  "yyyy-MM-dd'T'HH:mm:sszzz",
-  "yyyy-MM-dd'T'HH:mm:ss'.'fffzzz",
-]
-
-const ALL_FMTS = DATE_TIME_FMTS.toSeq & DATE_FMTS.toSeq
-
-proc parseDateTimeStr(
-    dateStr: string,
-    dateFmts: openarray[string]
-  ): DateTime {.inline, raises:[ValueError].} =
-
-  for fmt in dateFmts:
-    try: result = parse(dateStr, fmt)
-    except ValueError: discard
-
-  if not result.isInitialized:
-    raise newException(ValueError, "cannot parse date: " & dateStr )
-
-proc parseDate*(dateStr: string): DateTime =
-  parseDateTimeStr(dateStr, DATE_FMTS)
-
-proc parseDateTime*(dateStr: string): DateTime =
-  parseDateTimeStr(dateStr, DATE_TIME_FMTS)
-
-proc parseDateOrDateTime*(dateStr: string): DateTime =
-  parseDateTimeStr(dateStr, ALL_FMTS)
-
-func foldContentLine*(s: string): string =
-  result = ""
-  var rem = s
-  while rem.len > 75: # TODO: unicode multi-byte safe?
-    result &= rem[0..<75] & "\r\n "
-    rem = rem[75..^1]
-  result &= rem
-
-func unfoldContentLine*(s: string): string =
-  return s.multiReplace([("\r\n ", "")])
-
-template indexOfIt*(s, pred: untyped): int =
-  var result = -1
-  for idx, it {.inject.} in pairs(s):
-    if pred:
-      result = idx
-      break
-  result
--- a/src/vcard/vcard3.nim
+++ b/src/vcard/vcard3.nim
--- a/src/vcard/vcard4.nim
+++ b/src/vcard/vcard4.nim
--- a/tests/allen.foster.jpg.uri
+++ b/tests/allen.foster.jpg.uri
--- a/tests/allen.foster.v4.vcf
+++ b/tests/allen.foster.v4.vcf
@ -0,0 +1,212 @@
+BEGIN:VCARD
+VERSION:4.0
+PRODID:+//IDN bitfire.at//DAVx5/4.1.1-gplay ez-vcard/0.11.3
+UID:b7047a2e-c46b-47cb-af0b-94d354b7746a
+FN:Dr. Allen Foster
+N;SORT-AS=Foster,Allen,,,:Foster;Jack;John,Allen;Dr.;II
+NICKNAME:Jack Jr.
+NICKNAME;TYPE=work;PREF=1:Doc A
+TEL;TYPE=cell:+1 555-123-4567
+TEL;TYPE=cell:(555) 123-4567
+TEL;TYPE=work,voice;VALUE=uri:tel:+1-555-874-1234
+EMAIL;TYPE=work;PREF=2:jack.foster@company.test
+EMAIL;TYPE=home;PREF=1:allen@fosters.test
+SOURCE;VALUE=uri:https://carddav.fosters.test/allen.vcf
+KIND:individual
+REV:20220226T060828Z
+BDAY;ALTID=1;VALUE=date-and-or-time:--1224
+BDAY;ALTID=1;VALUE=text:Christmas Eve
+ANNIVERSARY:20140612T163000-0500
+GENDER:M;male
+MADE-UP-PROP:Sample value for my made-up prop.
+NOTE;LANG=en-us:This is an example\, for clarity; in text value cases the parser
+  will recognize escape values for '\,'\, '\\'\, and newlines. For example:\n	12
+ 3 Flagstaff Road\N	Placeville\, MA
+X-CUSTOM-EXAMPLE;PARAM="How one says, ^'Hello.^'";LABEL=^^top^nsecond line:This
+  is an example, for clarity; in straight value cases, the parser does not reco
+ gnize any escape values, as the meaning of the content is implementation-speci
+ fic.
+PHOTO;ALTID=1;VALUE=uri:https://tile.loc.gov/storage-services/service/pnp/
+ bellcm/02200/02297r.jpg
+URL:https://allen.fosters.test/
+PHOTO;ALTID=1:data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEFeAV4AAD/4QCuRXhpZgAASU
+ kqAAgAAAAHABIBAwABAAAAAQAAABoBBQABAAAAYgAAABsBBQABAAAAagAAACgBAwABAAAAAgAAADEB
+ AgANAAAAcgAAADIBAgAUAAAAgAAAAGmHBAABAAAAlAAAAAAAAAB4BQAAAQAAAHgFAAABAAAAR0lNUC
+ AyLjEwLjM0AAAyMDIzOjA0OjI1IDE2OjQzOjUyAAEAAaADAAEAAAABAAAAAAAAAP/hDM1odHRwOi8v
+ bnMuYWRvYmUuY29tL3hhcC8xLjAvADw/eHBhY2tldCBiZWdpbj0i77u/IiBpZD0iVzVNME1wQ2VoaU
+ h6cmVTek5UY3prYzlkIj8+IDx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1w
+ dGs9IlhNUCBDb3JlIDQuNC4wLUV4aXYyIj4gPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3Ln
+ czLm9yZy8xOTk5LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4gPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJv
+ dXQ9IiIgeG1sbnM6eG1wTU09Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEuMC9tbS8iIHhtbG5zOn
+ N0RXZ0PSJodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvc1R5cGUvUmVzb3VyY2VFdmVudCMiIHht
+ bG5zOmRjPSJodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLyIgeG1sbnM6R0lNUD0iaHR0cD
+ ovL3d3dy5naW1wLm9yZy94bXAvIiB4bWxuczp4bXA9Imh0dHA6Ly9ucy5hZG9iZS5jb20veGFwLzEu
+ MC8iIHhtcE1NOkRvY3VtZW50SUQ9ImdpbXA6ZG9jaWQ6Z2ltcDpmYzVkZDFkMC05ZmNiLTRhZjAtOG
+ UzNS1jMTMyMDU4NTUwMmEiIHhtcE1NOkluc3RhbmNlSUQ9InhtcC5paWQ6YmEyOThkOGYtMDY3NC00
+ ZDgzLWJhZGMtNWVkY2Y2OTg2NTBjIiB4bXBNTTpPcmlnaW5hbERvY3VtZW50SUQ9InhtcC5kaWQ6Nz
+ JjYWViYjMtMjkzMy00ZGJmLTg0M2EtYzYwYjBkZWYzMzdlIiBkYzpGb3JtYXQ9ImltYWdlL2pwZWci
+ IEdJTVA6QVBJPSIyLjAiIEdJTVA6UGxhdGZvcm09IldpbmRvd3MiIEdJTVA6VGltZVN0YW1wPSIxNj
+ gyNDU5MDQ2MjA4NjE3IiBHSU1QOlZlcnNpb249IjIuMTAuMzQiIHhtcDpDcmVhdG9yVG9vbD0iR0lN
+ UCAyLjEwIiB4bXA6TWV0YWRhdGFEYXRlPSIyMDIzOjA0OjI1VDE2OjQzOjUyLTA1OjAwIiB4bXA6TW
+ 9kaWZ5RGF0ZT0iMjAyMzowNDoyNVQxNjo0Mzo1Mi0wNTowMCI+IDx4bXBNTTpIaXN0b3J5PiA8cmRm
+ OlNlcT4gPHJkZjpsaSBzdEV2dDphY3Rpb249InNhdmVkIiBzdEV2dDpjaGFuZ2VkPSIvIiBzdEV2dD
+ ppbnN0YW5jZUlEPSJ4bXAuaWlkOjU2YzcxYmUyLWExNmMtNDE2OC1iNDA5LWI3YjRlMTgwZTFmMyIg
+ c3RFdnQ6c29mdHdhcmVBZ2VudD0iR2ltcCAyLjEwIChXaW5kb3dzKSIgc3RFdnQ6d2hlbj0iMjAyMy
+ 0wNC0yNVQxNjo0NDowNiIvPiA8L3JkZjpTZXE+IDwveG1wTU06SGlzdG9yeT4gPC9yZGY6RGVzY3Jp
+ cHRpb24+IDwvcmRmOlJERj4gPC94OnhtcG1ldGE+ICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
+ ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC
+ AgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgPD94cGFja2V0IGVuZD0idyI/Pv/iAjBJQ0Nf
+ UFJPRklMRQABAQAAAiBsY21zBEAAAG1udHJHUkFZWFlaIAfnAAQAGQAVACoAJWFjc3BNU0ZUAAAAAA
+ AAAAAAAAAAAAAAAAAAAAAAAAAAAAD21gABAAAAANMtbGNtcwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
+ AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABmRlc2MAAADMAAAAbmNwcnQAAAE8AAAANnd0cHQAAAF0AA
+ AAFGtUUkMAAAGIAAAAIGRtbmQAAAGoAAAAJGRtZGQAAAHMAAAAUm1sdWMAAAAAAAAAAQAAAAxlblVT
+ AAAAUgAAABwARwBJAE0AUAAgAGIAdQBpAGwAdAAtAGkAbgAgAEQANgA1ACAARwByAGEAeQBzAGMAYQ
+ BsAGUAIAB3AGkAdABoACAAcwBSAEcAQgAgAFQAUgBDAABtbHVjAAAAAAAAAAEAAAAMZW5VUwAAABoA
+ AAAcAFAAdQBiAGwAaQBjACAARABvAG0AYQBpAG4AAFhZWiAAAAAAAADzUQABAAAAARbMcGFyYQAAAA
+ AAAwAAAAJmZgAA8qcAAA1ZAAAT0AAAClttbHVjAAAAAAAAAAEAAAAMZW5VUwAAAAgAAAAcAEcASQBN
+ AFBtbHVjAAAAAAAAAAEAAAAMZW5VUwAAADYAAAAcAEQANgA1ACAARwByAGEAeQBzAGMAYQBsAGUAIA
+ B3AGkAdABoACAAcwBSAEcAQgAgAFQAUgBDAAD/2wBDAAoHBwgHBgoICAgLCgoLDhgQDg0NDh0VFhEY
+ Ix8lJCIfIiEmKzcvJik0KSEiMEExNDk7Pj4+JS5ESUM8SDc9Pjv/wgALCAEVAMgBAREA/8QAGwAAAQ
+ UBAQAAAAAAAAAAAAAAAAECAwQFBgf/2gAIAQEAAAAB5Fr1awQUQUAQVFFNHqq8aKrnACoCKKreNZvd
+ hACiigAAKqp5q3W7JEBVAUABQenmibfWtEFAUFEFUHN84TZ65iAAKoCiiit85Nrq2IKgENWa05QUVU
+ 82d0fRxoKhHztK3HPt3QFV0PnUnS77GgreTq6upo18vZeCq/K4p/S78aAM4vc3LUq0sXUkVVfD55d9
+ CmzEQXO5be27Er6vJ7Ogqq9vnWh6dFjIgVuN03bVKbR5Lo76qr2edaXpsWMiAc1as3bLZ8LWFcrmef
+ XvTWYgiKZJcp61+OBqjh8fn9705mGiAUcuOKzp2LIquHM4C76czEa2KOCvRq3Vi0Lcsr5Hvi4G56c3
+ DRjhEr4mpYguyKKqvi4G56emCICrjZs9nR2cLQQVXRcBd9QMAQUx+f6d+m/mqu3fFHQcLZ9RMFAq42
+ O6bQSFXFzWsq6vwc/qi4bMjPzUV0r5WsldG+9sWa/DSerGLn8oxVEnaOkc5Ho7U0OGX1kx+XyBXTE1
+ cVyucRzWN3g3+tGFw8Y96SyRDhwsUksujx83rZyfJxrK9tgYOVBsNiR0+IvricPz8ayPbZaiOckcTb
+ j5Ey3esnneXE6R5rZo5XNjrOuSuMjX9FPPMuJ73Jcja5RIqzrcsi4+36Cef5MMjnpciaqoxYWW5Xtz
+ On7Q4DHZI5VnYOVGRQlixMmUvrJ55jkkiq9Fc9WMrNlHy0m+tnn2MSSOlnrDpGqlJj3Wm0JfV04jny
+ SRz7KMJCOvVGOsW8af0qLjc2NXSPdKrbFaBI7LBroqgIAAAAAAB//EAC4QAAEDAgMIAQQDAQEAAAAA
+ AAEAAgMEERASIAUTFSExMjM0FCJAQUMjMEIkNf/aAAgBAQABBQJoCsgzMjHZWC5aOWjkuS5Llrsrcg
+ qaJszqWhiYjs+Jx4ZAuF064XTrhdOuF0y4ZSrhlKuF0q4ZSrhlKuGUq4ZSrhlKuGUq4ZSrhlKuGUq4
+ XSrhdKnbMprE/Qtn2C32eL7R/Yeiom3dlDB9mE4/QegWzvL9q/xntHTZ3k+1d2Htb02b3n+l80ca+d
+ EhUsKDg7+l3jTVstqOt8jY2z1r3lvXPHfMUyVzTDPvBrqHiOmTVstHUTYVkjnyMY56jo3qGiF/iRp9
+ AwowSU8oOvanpJq2brebNkOaSmaAGpuNQLxxTFNdfVUMbJTEKijEtXFQwQp8TQzTWXEX5gKYQgUCMJ
+ jaMO503MaApvAemzvfUvi01DDJCbgtLrZnNVM90rZS5joHp93w9FSMLYdM3gK2b76k8equjtMyDM34
+ 4VFGGGSnbIW0rVuw1scIA1TeB3Zs33wpPHqrmcoH8pJAI6ata0x1W9Oa4JW6DRqm8BH0bN99P7NNwp
+ 3tczNlDp8yytKicY3CsITKoPJmuAQdU3rm+TZvvp3ZhcLOSjnTg4rdqZuUtGYtaAixN7m5bht0GEIZ
+ l9Wif1zyj2b76d24WxssoU0THRRgBRtYUWssI2XbGMuUa5/Xd2bO95Ht1zVZcQQgSEwucoYyS2MRj5
+ Td5qqPWd02f739FXWbtRuLHsbFUt+C0qGljYpqyKnE9TJOgFBOHN01PquP0UHKt1S1EcKm2g5w/ITJ
+ Hwu4jOjUTvx6YQ1RjTJ436Kr1Ta1Hyqxg7uUtcGPlrZJF1w6LqmgFBgBR7Q3TFUvjTJmPwqvUtaKl9
+ vB/fVSbuA6ubRncV9SGsKkkIfVep0jp+VRhJ5K+bM52iIhrwxOvl0jAldxUB/mqvUzcoPLhUODC52Z
+ 7kNEfbiNF05ybgCt5noVD5cNqOswIoYlN7NV1dX5grmrreZYf1w+XDaz7zpyGJV7K+slNTV1QCk8f6
+ 6cEzYV/uooY/kOh3OopyahjJ4v87IANbhtH3V+UMP9I6i0ohBBXV1J4j2bGaTV4bSH/evziO86bhZ1
+ vE83QTOisn+L9expYoYcNp/+gvzi3yaLXW7C3SLLIoJhQcgpR/F+qAne4bVvxC6HXFvdm1FyzpyCvZ
+ ZkHFOkvH+mHy4bVuNoXQxsobB8+VzrYWxtdZAnCx/PJCya26kYAy38dOzMcNpi9Y5lk3RGPqsiFlQa
+ sqsiQnPVy5W+lWTHZS9wdCelOLy/MpwjtClCr5xJWZnFZiFnct65bxybM5p+Q9b9GdNqTf5Qs6oct8
+ 5bxOKfKHRte1FzL5m3acymZl+6/8QAMhAAAQIDBAoCAQMFAAAAAAAAAQACEBEhAyAxcRIiMDJBUWFy
+ gZFAsUITM6FSYoKi8P/aAAgBAQAGPwIznDBV+OQZ+ESQac1N5cSsX+1i/wBr8va/L2vy9rB3tbrvaw
+ d7W672t13tbrva3Xe1uu9rdd7W6fawd7W672sHe1g72jJrvaA6wceOOSFMfinJCAmMXALRBnL4rsos
+ 7/jOyQRX+Q+M7JBFeRstZ4CppHwuIVDPYuygU48thpONFJlApuRDBIcITXXYWjjyjaeNhX1GqwVEDi
+ Nh5EX7A3637RrhMShZsdgSjoA16okXqXyip8b1p2lBWWcHXnNGKqqGXVfuzUprRc8qlt4KdPGUBO8/
+ tMLLODr9OIhwTpKaqBXopIu0A6sq339pQzVlnB2V9r/ECiCiA0iUXHnW/adpWl1VlnA5X9FYqUJgyV
+ Vgqyv2naV0VlnA5XNUQxubs1RaypDCHC5adpWass4G/gnavC5gqrC/adphZZwOwfZ2LZkYmGKxVBNa
+ VoQtGVJ437TtMLLu2P6dnjxPJBwxC0i0TVHOCwnmpCruQWsfEJOOtetO0qSsu6/rOryWixuj1jNjpL
+ EDwta1dLld0XVCo6vW5a9phZd4+4mEmtmpN1Rdk7BUgb0jUKhha9hWasu8fcXZom/PRW7stDgVa9hR
+ HNWZ/ui5fpjAY3Zk8CqqROzbmrXsKITc4vceCLjx+HbA8GmDc4uHM3xtbTq0hFNzjo8viOyRzTQOcb
+ XO8FWU/wCVTaOyhUT1TG1zvDbOyXlF0qBsbX/uF4bZ2S8q2c53EUjaePq8Ns7JOzCaJ0JEbTx9fEcO
+ idmE3MRtPH1e1sENHHpewvYoo1wRd/TL7i/x9XjsKXnZQqZUVbVq/dT32Tpgy+oYLBYBboU5CqwasG
+ qgaqtat0LdC3Qt0LBMZo4cVufyt3/ZS0f5WhSvVfj4d8r/xAApEAACAQMCBwADAQADAAAAAAAAAREQ
+ ITFBUSBhcZGhsfAwgdHBQOHx/9oACAEBAAE/IXU2LEOBTcXsOwYZiaZFmRqWpbmWLHQ+5bZ9y1FuT7
+ nMn3Oh9zofcts+5altqRQ4T5lqdy7y0QUNbXpGJ0NuzhDfoD6H8LFJlsN+fWJYxjGIR1iUqN4JfqO6
+ k/QtSdkRYaDAQay+GSSaySTSSazRMWeuFvTdi1ESx2xbEVFpE/inhkngYQrcufowGok2bIO7mk7cEk
+ 8U/gbufR7p4hcv1keeKfzx58/RHuiT0Rbra3uhv8HYKkkcL0RnF1EKpUnLimvm/Q7noFtv1KPJgMnh
+ tJCRHTyxJkOCZ5dqFz7F1mhUDfGRptP3prEhdX+qHwyDG2eMbB5CUlhdQJ6pOEmgQ53RLWeJDQ3MtD
+ NlqfdoeB8KXNkm3YgWIxYaBMbJaMrBOjo4ki+CL7EEPcuRxnBb21TIX45S3Hwu68NRazEQckBqmStC
+ YF5WjkZSZbc+J8jYXzV/wOLOQVhGUkNFuxbhtymgO2hqPv6AhV2dkdMBuSWdTVS78KPibGmv+NxpS1
+ 1BazNKuUGgAtApITUHKy5KLg8PTLkQo24UfM2EU9plRcDyHHYKupEK3HlyLcN9RTTL76oSSEgkgoTW
+ c+JHyNhrfYi7z/KOsHnuKDUdkd5lMSTPxIWwjKMl3cLbEzMMFigQks54UfA2GlWppv790dYPNDyMaF
+ MqB6h5slWYM522G+7fIuGj2GQCOHJuOd0gfFGWmBJUQ0MbdjGKesJ7iEfA2JM2aKAsHgDpB5UkUasj
+ EoJKZjMujuaJBcYNiNiepC7IRwo+1sSSbN0VHifghCT+kIe1MM9A3dMR2Et8I/VhI+JDfNoYacsDum
+ uO8YmQ3MFls2qsxXSzZ3EyufmLDtLT6jabeiYVBQXSW68KPjbCcV5kaXyCxR2ccCbtGRjhHuuTOoyJ
+ kNXIsQuiGfmxb0aHgbhCy/Bofrj3Q/iDZZVHythmff0NL+YGFFhHOltandyeOLI22luWQSmuJBvM1L
+ KUtDtc9m6ImnkJSy1+xKDSiLwctiuUp6p6Cc4ufE2Lhn6+wWKL3hBeXZGNIEiJIjFhu/Y1lmgRfsls
+ tLohL3bZImJkCUDaV3gnkmJ7HU+JsNtREQkfkiPzXziFzvcaBVhUUQSpm0FuWTnmTJNXIiHRUVGtBP
+ mpaEwOnwh9TYb7RnhPdca0mN1gkbA0ioyx3uxsfAXALUJbiEJnaX61PAe6xTmM2GdCNKEjoDNaKjEH
+ kKVYQUTFnnYSUuspezxnuqILybYjKhCMBoiRcOio0NDFRBFwsaEcxpcn+iWJbT3WNDnQqaHMdjjXTW
+ RjKVzUColJFjA9GgXAoEIv2hp9JoQOXiqR9MUeFGJFfrjoVFYZEm7HLNCuJkTLIkZB9YJnyIVabKb2
+ r3Beg0MIgR5Y6ikCo06jShNC431OQkkB59H+jC0wOHajLPlgNiCojzxuao3hk0GhIohJBk6Aht23+g
+ si6klOb0Ykv7Yc1TQRmeyGmESZEhJDsNQ9xB4FEg5EJaHNQJXvi5Z916zBPK9A55QsuaQYGNcxdCYX
+ aZJohkskOJL7EMg4KRKHUNvgTjIeM3SLn2SVtzmJbwqmctPQaQSwlcgwPfSFodG4YIuAkPWAwOEmnf
+ Ak9BPq0jFNsmZXVhagqJqydtYuNP2hfLWOSbMfDH+hguOxovASP+hB/A/8hkSW0XTGrPaf9E8w8Wz/
+ AKPSuv0/6f4Rf9HYXOjJnrkbiTOFefUvYJ7ENK52RBOwM2FdoLXdjC6HNkrSh0l7/wDI/9oACAEBAA
+ AAEFWfpUjG1c/y/InCBKNszFj/AIjtwfXv6XJhrs8cSIIQVasIoenoFvkzL5NCrBxU3ELakaQpdEjY
+ XmSqnWYzVR9qL8HDZyK0M+3mUOLmSChUuis74DnOXiT3YPTWDJ6XOLnd7shbuefckppxh/8A/wD/AP
+ /EACgQAQACAgECBgMBAQEBAAAAAAEAESExQVFhEHGBkaGxwdHw8eEgQP/aAAgBAQABPxAsVTAsvUYa
+ UFLZjmtLh4mPuyU9oky90Bg07yr5PdK7veV0Necr/gfqf0/4ldD7wDkfeADYeKp+IZaPYfqb8fl/U1
+ 5/L+o9IedvxE4D7yjvKpt7yjp8ykp1lZnlEakRVfcaO4LHPdqXjo4FseUdesAC9BjgqE48gP1LK+jC
+ 4BKchz8Qrzb+9oFz57/if7Wf6mZBp7XzP9zP9DP9zMf5M/3Mf+xn+zn+lmL8mH/Swb92BiWt3rDmKv
+ YKHeo1i/RIVckpzvj3I6wjL8e8XvLl8y5jjxBly3iX4MpcFhF0VM0TLQua24YRnrLK1qa+SH3URVOf
+ x79otUEHtI5lkslwZcuXB8Fy5cuC6+Icy4BuGVMDfGWZXv288YlqoXj8x0wopq7Y7IWmNG8JcuD4lw
+ ZcuX4FlwbhFqHWDCMOXCeaEKtrb4nwX3MRdafb9CPLziO4vEuXL8Fy4MuXLly5UIOZiXBhHUfeiRCW
+ NjnieQg/MwKlTPl/2+0WXwLzLgxZcGXBm1OVn2MzLIuhoH3SJ1d/1i533ApcGDiXBloMNx0jo+1FaB
+ QHP5R170XkfywgE6RTaDL8UoC56xk03GnmsMHd2nL6suoxyWnS4odHgyjCCmGqgYVPmDLly4MGLEfM
+ AQLtFB6qeG/lPshUOh8RpfWBK7x8LlzEK0XRAKsdE6QHTXuNyZluHHaWwgJvpvyiYSKjkkNp0vMN1w
+ gy4MIRyt0tLGje4pUrNues+MfZCpNuX6P7ivwrl+BFXAB3KHbtpg+jEdIAmgTqwXTLaqzoiiy02svM
+ Co2PcqEGEIZpoF0iLE9Qing0hdI4qlPOCC3DZs+sLisDbwM14hXdKz4jorhlgL4IADV851E4SebFDI
+ lr8dIIYVsOntC1QUKmvvwIah4OB/SlSc2PqXwHGGSf0d5dx/8AH5jQMxEiqR2RhMAWcQAw2DjzlONi
+ oawV0+CLRluwelzX8WTTGZGxauWJ5q+iZBhBhCOL+7lOKLc0fow1DZd80x8L8HOJRARYADmmZl4Itf
+ GWhx+IHZ8WIql7uswLh0M+ZgGWxWW4qHVd8BrZ3zcCR1gQhCHh/rdUKTSOc1Rx03K5+PwYrgX/AEYi
+ 1FuMYa8AbRenUS/xBDpUtTkhmK2qD43ZpTclQwOZmEdAMugftnHgQ8V8VXzeaOLXtZxh/TTNUF/yYj
+ FZfgRuWcShiRwMRtYrgsszscMWbhqN7WFaAV27zR84qrlOpXJkItsLaS2GOo3LhOITaf0uqUXnzALD
+ 7QXs/wAMvS4bD+KgpHeWHVnaMXU5uYRvQEFRXeAn7iLZeqH8uPLwjQp/veUhMFbUV17xQc9Zm7uXEM
+ vdpBs0mgIhg3xm6f1AtENWohEydda3ziy0CutQzVp2u42jTKTeYvr9yOiMFZ6U/mb91X4sdm58v9TZ
+ jqLhBDySlEqbchnGXVzEFABMNhiUigV94JSKrqOKC3EWFwEwzslGVdiBCBO0oaKic8wDpKuEcf8AZy
+ hCvZq+Q/7EFm0hBQEzDv8AqO4+FwhAlpA7G7BrqNwcD0O5kSeswSklTAbauolGt5eosL6F3fCroXE0
+ 7pJUPAjgDL/oltY276f1ymboisVD3QTwfAxCEvLEocluDvESCX5nMCyGfuDccshyJTy183MKJzdfoY
+ +JdMxlAehr7gNHlflEZbiZ5pbVePWcSvAgnJ1+RGi7Ue1B+GeWv7tfmVrWvAqdGIyoEbUPDOvSU+vV
+ rV7dIYlJeUiAWVD5taA2vMcQcXTVd584TTAoYL5gLgU67hRLChbapnRq30fuBUrAvOX5IKI3ef8AfA
+ nCcP8AtTHdoKR6o4nJe4TT4dhmfMuoxNVPS3tBFL8Z5esYqTatrCzDjVlDAQG3XeAN6Kw5ouNwzPJr
+ 0idSBhMvmFAV1GTvFdq5YuZ2lLSTOu0oEFz5PJlTALXTBAUB0jc/s9UEuGRTtw/EdI6LYGvAwX8XEe
+ UfrsVBuEIwTiq4blht4i4MNqIH9SvH8xj3YCVzqsAQqlmYaSg3L1sDXVlxfK6OCWFIqF0M/Vz+j1QR
+ 7OUzhxLQ016CGvDEO+dbGXXo9IsDr4CBiO6jrASwY1vcpQJIUETrZ/YZXDQKIvsKSlAUSoM30g8aeY
+ msaizl3BwlKETECduWBFa7dId6CLX2i/h5RJRKvWpTNfB7JVPgrNOk36OvVmXLUpX4cBiasJyCfqHm
+ vmWu5zMMxHcRXeWmZRLeCZMOaxDVTmWNu5Q6ix6su6VX1HbLjHH0PG9VezbiTIjrtjqOoaRUYnbGT6
+ 5gVz6TFr1OcRmlrv4LpL9sI24lE7CXFG+kCheJRhRlCqmO6qfeWHWGx/OIPT+tHfgcuPYV6i6xjHCa
+ zQzNwRi8AexDiIvaGoMQxjMsyRKXTLgXiG0VlAQ95ky0dpRyVAG0/VLBzlUOtZfZGf1gOwY78CkbW8
+ wOJd0mGGFgXH2CNRC+cJz5VLuidf4BMoAmTMtUaiNukcq3Uo4izKB1S4ygGjMvdOICwLxe0I65Fx0n
+ SDw2L+XwYJjpfcManzIlzidzayWB1MFGpV0SlwLNSuSx23LhprEaKWI3Ql6MxxEYkxTcXW93pAXzAe
+ o39EUBXHhKV6tPgxKO7BiFrXWWSoc4esy3ay4q7HmOSMYG2Acs2kWqHEAFSxQI8S+JphTBgatZjUcq
+ Z9v6oCnetS7KvKtnhbiNycM7XuC09JvAKgxBaui3sRu+NnEuuJmHWACJIvm+0drE8qjiOIWGszRsS1
+ yCzAoqrmesU5fuhUJZznwClEBQaqMULPuTIwxCa1BTNbel4lgUfKCdxOEKtytlgchitMFKHvG6wC5m
+ jIt2yntEJVyhHVNGI6w7X8V8KmiBNSDR9JcjQQMwwJyVEtBKWSww1fa6l+EVTJpQvvLWS4VQLqhoIt
+ sIzAGIesZapdHEKFYKyyh1RKW/RBrI9KlNJdpEWUVCpX5fqvmOKtJg9QjvwptaC4WV06SpNS4QxqBQ
+ ouAZujFHkeYPrGjEHdZOkYLPaI6SybkgyXx1ELC3NTCY7KwxDQb7QH2ECy34qiU+WFcjCg6ynUrdqK
+ Fc5IJpoF1b6gN1FmW9iBo0g00Dh7jElGXRS1gRdqi1oOlqNT7n7he8X8biG8taXzELtVZzw48qtzvS
+ F98TFxne+CpDW7xfct6PyftMi3+TNkADiBqVCjpAe+9n5ivzDwTqub5Ym8Q9mPiGAVF22/E6DCRB5r
+ GAaDKL5DfxBTTUUbUr1Zb/APP/AP/Z
+END:VCARD
--- a/tests/jdb.vcf
+++ b/tests/jdb.vcf
@ -1,6 +1,6 @@
 BEGIN:VCARD
-PRODID:-//CyrusIMAP.org//Cyrus 3.7.0-alpha0-927-gf4c98c8499-fm-202208..//EN
 VERSION:3.0
+PRODID:-//CyrusIMAP.org//Cyrus 3.7.0-alpha0-927-gf4c98c8499-fm-202208..//EN
 UID:cdaf67dc-b702-41ac-9c26-bb61df3032d2
 N:Bernard;Jonathan;;;
 FN:Jonathan Bernard
--- a/tests/tlexer.nim
+++ b/tests/tlexer.nim
@ -1,2 +1,6 @@
 import unittest
-import vcard/private/lexer
+import ./vcard/private/lexer
+
+suite "vcard/private/lexer":
+  test "private lexer tests":
+    runVcardLexerPrivateTests()
--- a/tests/tvcard3.nim
+++ b/tests/tvcard3.nim
@ -1,24 +1,42 @@
-import options, unittest, vcard3, zero_functional
+import options, unittest, zero_functional
+
+import ./vcard
+import ./vcard/vcard3

 suite "vcard/vcard3":

-  let testVCard =
-    "BEGIN:VCARD\r\n" &
-    "VERSION:3.0\r\n" &
-    "FN:Mr. John Q. Public\\, Esq.\r\n" &
-    "N:Public;John;Quinlan;Mr.;Esq.\r\n" &
-    "END:VCARD\r\n"
+  test "vcard3/private tests":
+    runVcard3PrivateTests()

-  test "minimal VCard":
-    let vc = parseVCard3(testVCard)[0]
+  let jdbVCard = readFile("tests/jdb.vcf")
+
+  test "parseVCard3":
+    check parseVCards(jdbVCard).len == 1
+
+  test "parseVCard3File":
+    check parseVCardsFromFile("tests/jdb.vcf").len == 1
+
+  # TODO: remove cast after finishing VCard4 implementation
+  let jdb = cast[VCard3](parseVCards(jdbVCard)[0])
+
+  test "email is parsed correctly":
    check:
-      vc.n.family[0] == "Public"
-      vc.n.given[0] == "John"
-      vc.fn.value == "Mr. John Q. Public\\, Esq."
+      jdb.email.len == 7
+      jdb.email[0].value == "jonathan@jdbernard.com"
+      jdb.email[0].emailType.contains("pref")
+      jdb.email[0].emailType.contains("home")
+      jdb.email[1].value == "jdb@jdb-software.com"
+      jdb.email[1].emailType.contains("work")
+      jdb.email[2].group.isSome
+      jdb.email[2].group.get == "email2"
+      jdb.email[6].value == "jbernard@vectra.ai"
+      jdb.email[6].emailType.contains("work")

-  test "serialize minimal VCard":
-    let vc = parseVCard3(testVCard)[0]
-    check $vc == testVCard
+  test "tel is parsed correctly":
+    check:
+      jdb.tel.len == 2
+      jdb.tel[0].value == "(512) 777-1602"
+      jdb.tel[0].telType.contains("CELL")

  test "RFC2426 Author's VCards":
    let vcardsStr =
@ -47,18 +65,9 @@ suite "vcard/vcard3":
      "EMAIL;TYPE=INTERNET:howes@netscape.com\r\n" &
      "END:vCard\r\n"

-    let vcards = parseVCard3(vcardsStr)
+    let vcards = parseVCards(vcardsStr)
    check:
      vcards.len == 2
-      vcards[0].fn.value == "Frank Dawson"
-      vcards[0].email.len == 2
-      (vcards[0].email --> find(it.emailType.contains("PREF"))).isSome
-
-  test "Jonathan Bernard VCard":
-    #const jdbVcard = readFile("tests/jdb.vcf")
-    let jdb = parseVCard3File("tests/jdb.vcf")[0]
-    check:
-      jdb.email.len == 7
-      jdb.email[0].value == "jonathan@jdbernard.com"
-      jdb.email[0].emailType.contains("pref")
-      jdb.fn.value == "Jonathan Bernard"
+      cast[VCard3](vcards[0]).fn.value == "Frank Dawson"
+      cast[VCard3](vcards[0]).email.len == 2
+      (cast[VCard3](vcards[0]).email --> find(it.emailType.contains("PREF"))).isSome
--- a/tests/tvcard4.nim
+++ b/tests/tvcard4.nim
@ -0,0 +1,250 @@
+import std/[options, strutils, tables, unittest]
+import zero_functional
+
+import ./vcard
+import ./vcard/vcard4
+
+suite "vcard/vcard4":
+
+  test "vcard4/private tests":
+    runVcard4PrivateTests()
+
+  let v4ExampleStr = readFile("tests/allen.foster.v4.vcf")
+
+  let testVCardTemplate =
+    "BEGIN:VCARD\r\n" &
+    "VERSION:4.0\r\n" &
+    "$#" &
+    "END:VCARD\r\n"
+
+  test "parseVCard4":
+    check parseVCards(v4ExampleStr).len == 1
+
+  test "parseVCard4File":
+    check parseVCardsFromFile("tests/allen.foster.v4.vcf").len == 1
+
+  # TODO: remove cast after finishing VCard4 implementation
+  let v4Ex = cast[VCard4](parseVCards(v4ExampleStr)[0])
+
+  test "RFC 6350 author's VCard":
+    let vcardStr =
+      "BEGIN:VCARD\r\n" &
+      "VERSION:4.0\r\n" &
+      "FN:Simon Perreault\r\n" &
+      "N:Perreault;Simon;;;ing. jr,M.Sc.\r\n" &
+      "BDAY:--0203\r\n" &
+      "ANNIVERSARY:20090808T1430-0500\r\n" &
+      "GENDER:M\r\n" &
+      "LANG;PREF=1:fr\r\n" &
+      "LANG;PREF=2:en\r\n" &
+      "ORG;TYPE=work:Viagenie\r\n" &
+      "ADR;TYPE=work:;Suite D2-630;2875 Laurier;\r\n" &
+      " Quebec;QC;G1V 2M2;Canada\r\n" &
+      "TEL;VALUE=uri;TYPE=\"work,voice\";PREF=1:tel:+1-418-656-9254;ext=102\r\n" &
+      "TEL;VALUE=uri;TYPE=\"work,cell,voice,video,text\":tel:+1-418-262-6501\r\n" &
+      "EMAIL;TYPE=work:simon.perreault@viagenie.ca\r\n" &
+      "GEO;TYPE=work:geo:46.772673,-71.282945\r\n" &
+      "KEY;TYPE=work;VALUE=uri:\r\n" &
+      " http://www.viagenie.ca/simon.perreault/simon.asc\r\n" &
+      "TZ:-0500\r\n" &
+      "URL;TYPE=home:http://nomis80.org\r\n" &
+      "END:VCARD\r\n"
+
+    let vcards = parseVCards(vcardStr)
+    check vcards.len == 1
+    let sp = cast[VCard4](vcards[0])
+    check:
+      sp.fn.len == 1
+      sp.fn[0].value == "Simon Perreault"
+      sp.gender.isSome
+      sp.gender.get.sex == some(VC4_Sex.Male)
+      sp.gender.get.genderIdentity.isNone
+      sp.lang.len == 2
+      sp.lang --> map(it.value) == @["fr", "en"]
+
+  test "custom properties are serialized":
+    let email = newVC4_Email(
+      value ="john.smith@testco.test",
+      types = @["work", "internet"],
+      params = @[("PREF", @["1"]), ("X-ATTACHMENT-LIMIT", @["25MB"])])
+
+    check serialize(email) ==
+      "EMAIL;X-ATTACHMENT-LIMIT=25MB;TYPE=work,internet;PREF=1:john.smith@testco.test"
+
+  test "can parse properties with escaped characters":
+    check v4Ex.note.len == 1
+    let note = v4Ex.note[0]
+
+    check note.value ==
+      "This is an example, for clarity; in text value cases the parser " &
+      "will recognize escape values for ',', '\\', and newlines. For " &
+      "example:" &
+      "\n\t123 Flagstaff Road" &
+      "\n\tPlaceville, MA"
+
+  test "can parse parameters with escaped characters":
+    let prop = v4Ex.customProp("X-CUSTOM-EXAMPLE")[0]
+    check prop.value ==
+      "This is an example, for clarity; in straight value cases, the parser " &
+      "does not recognize any escape values, as the meaning of the content " &
+      "is implementation-specific."
+    let param1 = prop.params --> filter(it.name == "PARAM")
+    let label = prop.params --> filter(it.name == "LABEL")
+    check:
+      param1.len == 1
+      param1[0].values == @["How one says, \"Hello.\""]
+      label.len == 1
+      label[0].values == @["^top\nsecond line"]
+
+  test "Data URIs are parsed correctly":
+    let expectedB64 = readFile("tests/allen.foster.jpg.uri")
+
+    check:
+      v4Ex.photo.len == 2
+      v4Ex.photo[0].altId == some("1")
+      v4Ex.photo[0].value ==
+        "https://tile.loc.gov/storage-services/service/pnp/bellcm/02200/02297r.jpg"
+      v4Ex.photo[0].valueType == some("uri")
+      v4Ex.photo[1].altId == some("1")
+      v4Ex.photo[1].value == expectedB64
+      v4Ex.photo[1].valueType.isNone
+
+  test "URI-type properties are parsed correctly":
+    # Covers SOURCE, PHOTO, IMPP, GEO, LOGO, MEMBER, SOUND, URL, FBURL,
+    # CALADRURI, and CALURI
+    check:
+      v4Ex.source.len == 1
+      v4Ex.source[0].value == "https://carddav.fosters.test/allen.vcf"
+      v4Ex.source[0].valueType == some("uri")
+      v4Ex.url.len == 1
+      v4Ex.url[0].value == "https://allen.fosters.test/"
+
+  test "URI-type properties are serialized correctly":
+    # Covers SOURCE, PHOTO, IMPP, GEO, LOGO, MEMBER, SOUND, URL, FBURL,
+    # CALADRURI, and CALURI
+    let src = newVC4_Source(value="https://carddav.example.test/john-smith.vcf")
+    check serialize(src) == "SOURCE:https://carddav.example.test/john-smith.vcf"
+
+  test "Single-text properties are parsed correctly":
+    # Covers KIND, XML, FN, NICKNAME, EMAIL, LANG, TZ, TITLE, ROLE, ORG, NOTE,
+    # PRODID, and VERSION
+    check:
+      v4Ex.kind.isSome
+      v4Ex.kind.get.value == "individual"
+      v4Ex.nickname.len == 2
+      v4Ex.nickname[0].value == @["Jack Jr."]
+      v4Ex.nickname[1].value == @["Doc A"]
+      v4Ex.fn.len == 1
+      v4Ex.fn[0].value == "Dr. Allen Foster"
+      v4Ex.email.len == 2
+      v4Ex.email[0].value == "jack.foster@company.test"
+      v4Ex.email[0].types == @["work"]
+
+  test "URI or Text properties are parsed correctly":
+    # Covers TEL, RELATED, UID, KEY
+    check:
+      v4Ex.tel.len == 3
+      v4ex.tel[0].types == @[$VC4_TelType.ttCell]
+      v4Ex.tel[0].value == "+1 555-123-4567"
+      v4Ex.tel[2].types == @[$VC4_TelType.ttWork,$VC4_TelType.ttVoice]
+      v4Ex.tel[2].valueType == some($vtUri)
+      v4Ex.tel[2].value == "tel:+1-555-874-1234"
+
+  test "N is parsed correctly":
+    check:
+      v4Ex.n.isSome
+      v4Ex.n.get.given == @["Jack"]
+      v4Ex.n.get.family == @["Foster"]
+      v4Ex.n.get.additional == @["John", "Allen"]
+      v4Ex.n.get.prefixes == @["Dr."]
+      v4Ex.n.get.suffixes == @["II"]
+
+  test "BDAY is parsed correctly":
+    check:
+      v4Ex.bday.isSome
+      v4Ex.bday.get.value == "--1224"
+      v4Ex.bday.get.year.isNone
+      v4Ex.bday.get.month == some(12)
+      v4Ex.bday.get.day == some(24)
+
+  test "ANNIVERSARY is parsed correctly":
+    check:
+      v4Ex.anniversary.isSome
+      v4Ex.anniversary.get.value == "20140612T163000-0500"
+      v4Ex.anniversary.get.year == some(2014)
+      v4Ex.anniversary.get.hour == some(16)
+      v4Ex.anniversary.get.minute == some(30)
+      v4Ex.anniversary.get.timezone == some("-0500")
+
+  test "GENDER is parsed correctly":
+    check:
+      v4Ex.gender.isSome
+      v4Ex.gender.get.sex == some(VC4_Sex.Male)
+      v4Ex.gender.get.genderIdentity == some("male")
+
+#[
+  test "CATEGORIES is parsed correctly":
+  test "REV is parsed correctly":
+  test "CLIENTPIDMAP is parsed correctly":
+]#
+
+  test "unknown properties are parsed correctly":
+
+    check v4Ex.customProp("MADE-UP-PROP").len == 1
+    let madeUpProp = v4Ex.customProp("MADE-UP-PROP")[0]
+    check:
+      madeUpProp.name == "MADE-UP-PROP"
+      madeUpProp.value == "Sample value for my made-up prop."
+
+  let cardWithAltBdayStr = testVCardTemplate % [(
+    "BDAY;VALUE=text;ALTID=1:20th century\r\n" &
+    "BDAY;VALUE=date-and-or-time;ALTID=1:19650321\r\n"
+  )]
+
+  test "single-cardinality properties allow multiples with ALTID":
+    check parseVCards(cardWithAltBdayStr).len == 1
+
+  let hasAltBdays = cast[VCard4](parseVCards(cardWithAltBdayStr)[0])
+
+  test "properties with cardinality 1 and altids return the first found by default":
+    check:
+      hasAltBdays.bday.isSome
+      hasAltBdays.bday.get.value == "20th century"
+      hasAltBdays.bday.get.year.isNone
+
+  test "allAlternatives":
+    check:
+      hasAltBdays.content.len == 3
+      hasAltBdays.bday.isSome
+
+    let allBdays = allAlternatives[VC4_Bday](hasAltBdays)
+    check:
+      allBdays.len == 1
+      allBdays.contains("1")
+      allBdays["1"].len == 2
+
+    let bday0 = allBdays["1"][0]
+    check:
+      bday0.value == "20th century"
+      bday0.year.isNone
+      bday0.month.isNone
+      bday0.day.isNone
+      bday0.hour.isNone
+      bday0.minute.isNone
+      bday0.second.isNone
+      bday0.timezone.isNone
+
+    let bday1 = allBDays["1"][1]
+    check:
+      bday1.value == "19650321"
+      bday1.year == some(1965)
+      bday1.month == some(3)
+      bday1.day == some(21)
+      bday1.hour.isNone
+      bday1.minute.isNone
+      bday1.second.isNone
+
+  test "PREF ordering":
+    check:
+      v4Ex.nickname --> map(it.value) == @[@["Jack Jr."], @["Doc A"]]
+      v4Ex.nickname.inPrefOrder --> map(it.value) == @[@["Doc A"], @["Jack Jr."]]
--- a/vcard.nimble
+++ b/vcard.nimble
@ -1,6 +1,6 @@
 # Package

-version       = "0.1.2"
+version       = "0.2.0"
 author        = "Jonathan Bernard"
 description   = "Nim parser for the vCard format version 3.0 (4.0 planned)."
 license       = "MIT"
Author	SHA1	Message	Date
Jonathan Bernard	ce70e5ddd4	Bump version in prep for release.	2023-05-03 07:11:32 -05:00
Jonathan Bernard	935f1bae2f	WIP documentation - The documentation is cluttered enough as it is with the large number of procedures supporting vCard 3 and 4. Split common out into the publicly exposed bits and the private internals. This makes it obvious which common functionality a client can expect to have exposed on the main vcard module. - Add documentation (WIP) on the vcard3 module.	2023-05-03 02:16:18 -05:00
Jonathan Bernard	0ec1856d1b	lexer: Add comments documenting the implementation and expected behavior.	2023-05-03 01:05:02 -05:00
Jonathan Bernard	ddddcf0af9	vcard4: Use VC_Param instead of VCParam for consistency.	2023-05-03 00:29:55 -05:00
Jonathan Bernard	5565087359	common: Reformat for clarity, documentation. - Consolidate `vcard/private/util` into `vcard/private/common`.	2023-05-03 00:28:26 -05:00
Jonathan Bernard	98c300fee2	vcard4: Complete implementation. - Parsers and serializers are now present for all property types. - Tests exist to cover parsing for most value types. Many property types share the same parsing logic based on their value type. We have created unit tests to cover each value type, not neccesarily all properties individually.	2023-05-02 22:37:23 -05:00
Jonathan Bernard	daa58518e3	vcard3: Unify with VCard4 implementation. - Unify the naming pattern of types and enums. Specifically: - use `VC_Param` instead of `VCParam`. The goal here is to make the important information (Param, Source, PropertyName, etc.) easy to see at a glance while preserving a prefix that allows multiple implementation to coexist (VC3_Source vs. VC4_Source) and also be easily distinguishable at a glance. - use `pnName` instead of `cnName`. The VCard standard refers to each line of data as a "content line," so the original name of each was "Content." However, the spec more commonly refers to each piece of data as a "property." There is a 1-to-1 mapping of content lines to property instances, but property is a more accurate name. - Introduce the idea of property cardinality to the VCard3 implementation. The spec does not tightly define property cardinality, but implies it with statements like "if the NAME type is present, then its value is the displayable, presentation text associated..." (emphasis added). Similar language implies some properties must be present exactly once (FN, N, VERSION) or at most once (NAME, PROFILE, SOURCE, BDAY, CATEGORIES, PRODID, REV, SORT-STRING, UID). Any other properties are assumed to be allowed any number of times (including 0). In the case of a VCard that contains multiple instances of properties expected to be singular, the parser will still parse and store these properties. They can be accessed via the `vcard#allPropsOfType` function. For example: # vc3 is a VCard3 allPropsOfType[VC3_N](vc3) If we see over the course of time that other implementations regularly use multiple instances of properties we have expected to be singular, we are open to changing the contract to treat them so (though this may be a breaking change). - Refactor the VCard3 implementation to use generated property accessors, following a similar pattern to the new VCard4 implementation. - Remove the accessor definitions that allow access via the content seq directly (`vc3.content.name` for example). There really isn't a reason for this use-case and the library is simpler without exposing this.	2023-05-02 22:36:27 -05:00
Jonathan Bernard	8e25c3d100	lexer, common: More descriptive error messages. The lexer now tracks the data that has been read since the start of the current line. While this may have use in parsers, the immediate use is by the common error reporting procedure. The `common#error` procedure already reports the column and line number where an error occurs. The `common#expect` function is broadly used by parsers and generates the majority of parser errors. It now uses the lexer's record of the current line to format its error message with a direct pointer to the location of the unmet expectation.	2023-05-02 22:11:00 -05:00
Jonathan Bernard	71107dda1c	lexer: Add readLen and readRunesLen. Convenience methods for cases where a parser knows it wants to read multiple bytes or runes from the input stream.	2023-05-02 22:05:00 -05:00
Jonathan Bernard	cf4c14f9f8	Makefiles: update tests to ignore the BareExcept warning. The `unittest` library still contains a number of bare exceptions, which now result in warnings from the compliler. Patching the standard library to remove these warnings is outside the scope of this project, so we're going to ignore these warnings.	2023-05-02 21:32:30 -05:00
Jonathan Bernard	31f47f60c2	Makefile: Updates to testing targets.	2023-04-23 22:55:41 -05:00
Jonathan Bernard	f59403ad72	WIP - initial VCard4 implementation.	2023-04-23 21:56:15 -05:00
Jonathan Bernard	7d642adf2d	RFC 6868: erratta for VCard4 parameter escaping mechanism.	2023-04-23 21:56:15 -05:00
Jonathan Bernard	3f1efe9e85	Makefile.	2023-04-23 21:56:15 -05:00
Jonathan Bernard	6bbcd9b6a3	lexer: Support multiple nested bookmarks.	2023-04-23 21:56:10 -05:00
Jonathan Bernard	8e58189a8b	Re-organizing code in preparation for v4.0 implementation.	2023-04-23 21:56:04 -05:00
Jonathan Bernard	9d030132de	Better ignore pattern for test artifacts.	2023-04-23 21:55:53 -05:00
Jonathan Bernard	68554920e5	Fix bug in parsing TEL content. Rework unit tests. - newVC3_Tel was not assigning the value provided to the constructed object. - Private unit tests were run every time the code was compiled due to how the unittest library works. These now only run as part of the unit tests with `nimble test`.	2023-04-16 03:34:14 -05:00
Jonathan Bernard	7b71cb2dfe	Extract example from the README to a runnable location.	2023-04-16 03:31:37 -05:00