Compare commits

...

17 Commits
v1.7 ... main

Author SHA1 Message Date
Jonathan Bernard
6d202a49d7 Clean up whitespace. Fix Main-Class manifest entry. 2016-01-26 01:47:01 -06:00
Jonathan Bernard
7ef580ba17 Bugfix in LiterateMarkdownGenerator. Documentation moved to doc.jdb-labs.com. 2015-05-12 20:30:23 -05:00
Jonathan Bernard
467267883e Removed old ant build files. 2015-02-05 00:31:27 -06:00
Jonathan Bernard
8de942af1a Removed local lib copies (using gradle now). 2015-02-05 00:30:33 -06:00
Jonathan Bernard
f98dd382fe Moved to gradle build system. 2015-02-05 00:25:56 -06:00
Jonathan Bernard
a9e2bc0d46 Merge branch 'master' of ssh://dev.jdb-labs.com/home/git/projects/jdb-labs/jlp 2015-02-05 00:12:56 -06:00
Jonathan Bernard
3609348a94 Added @param directive. 2015-02-05 00:12:51 -06:00
Jonathan Bernard
d58376d025 Updated Processor utility functions to add support for backslashes in paths. 2014-06-21 19:49:55 +00:00
Jonathan Bernard
caf96db6c4 Fixed broken links in documentation. 2013-08-13 12:15:12 -05:00
Jonathan Bernard
31cf009598 Updating README with a link to the generated documentaion. 2013-08-13 12:02:09 -05:00
Jonathan Bernard
d937e591cc Modified README to play nice with GitHub instead of JLP. 2013-08-13 11:30:59 -05:00
Jonathan Bernard
477233d020 Minor documentation formatting. 2013-08-13 09:52:10 -05:00
Jonathan Bernard
928c1ebe3b Formatting documentation. 2012-10-22 20:22:10 -05:00
Jonathan Bernard
e5cd57414e Added thoughts about general direction. 2012-02-05 21:20:35 -06:00
Jonathan Bernard
e1226642cd Tidying up minor docs. 2012-01-30 14:36:51 -06:00
Jonathan Bernard
71975412d2 Now using github to track project issues. 2012-01-30 14:11:56 -06:00
Jonathan Bernard
fcdf829cb1 Updated issues. 2012-01-30 14:00:01 -06:00
61 changed files with 171 additions and 474 deletions

1
.gitignore vendored
View File

@ -1,3 +1,4 @@
.gradle/
docs/jlp-docs/ docs/jlp-docs/
build/ build/
.*.sw? .*.sw?

View File

@ -1,5 +1,8 @@
# J Literate Programming # J Literate Programming
* [Source](https://git.jdb-labs.com/jdb-labs/jlp)
* [Annotated Source and Documentation](https://doc.jdb-labs.com/jlp/current/)
## Overview ## Overview
*Jonathan's Literate Programming* is my take on literate programming. *Jonathan's Literate Programming* is my take on literate programming.
This project grew out of a desire for a documentation system that: This project grew out of a desire for a documentation system that:
@ -57,14 +60,19 @@ This project is in its infancy and some of the larger goals are still unmet:
## Project Architecture ## Project Architecture
JLP processes it's own documentation. The latest documentation is available at
https://doc.jdb-labs.com/jlp/current/
Below are some starting points.
### Control and Flow ### Control and Flow
* [JLPMain](jlp://jlp.jdb-labs.com/JLPMain) * [JLPMain](https://doc.jdb-labs.com/jlp/current/src/main/groovy/com/jdblabs/jlp/JLPMain.groovy.html)
The entry point to the JLP executable. Parses the command line input and The entry point to the JLP executable. Parses the command line input and
sets up the processor. sets up the processor.
* [Processor](jlp://jlp.jdb-labs.com/Processor) * [Processor](https://doc.jdb-labs.com/jlp/current/src/main/groovy/com/jdblabs/jlp/Processor.groovy.html)
The Processor processes one batch of input files to create a set of output files. The Processor processes one batch of input files to create a set of output files.
It holds the intermediate state needed by the generators and coordinates the It holds the intermediate state needed by the generators and coordinates the
@ -72,39 +80,36 @@ This project is in its infancy and some of the larger goals are still unmet:
processor only generates HTML documentation and will likely be renamed in processor only generates HTML documentation and will likely be renamed in
the future to reflect this. the future to reflect this.
* [JLPBaseGenerator](jlp://jlp.jdb-labs.com/JLPBaseGenerator) * [JLPBaseGenerator](https://doc.jdb-labs.com/jlp/current/src/main/groovy/com/jdblabs/jlp/JLPBaseGenerator.groovy.html)
The Generator processes one input file. It parses the AST for the input file The Generator processes one input file. It parses the AST for the input file
and emits the final documentation for the file. JLPBaseGenerator and emits the final documentation for the file. JLPBaseGenerator
implementations are expected to be tightly coupled to Processor implementations are expected to be tightly coupled to Processor
implementations. implementations.
* [LiterateMarkdownGenerator](jlp://jlp.jdb-labs.com/LiterateMarkdownGenerator) * [LiterateMarkdownGenerator](https://doc.jdb-labs.com/jlp/current/src/main/groovy/com/jdblabs/jlp/LiterateMarkdownGenerator.groovy.html)
This implemetation of JLPBaseGenerator generates literate-style This implemetation of JLPBaseGenerator generates literate-style
documentation (as opposed to API-style), using [Markdown] to format the documentation (as opposed to API-style), using
[Markdown](http://daringfireball.net/projects/markdown/) to format the
documentation blocks. documentation blocks.
[Markdown]: http://daringfireball.net/projects/markdown/
### Parsing ### Parsing
* [JLPParser](jlp://jlp.jdb-labs.com/JLPParser) * [JLPParser](https://doc.jdb-labs.com/jlp/current/src/main/groovy/com/jdblabs/jlp/JLPParser.groovy.html)
A very simple interface for parsing JLP input. A very simple interface for parsing JLP input.
* [JLPPegParser](jlp://jlp.jdb-labs.com/JLPPegParser) * [JLPPegParser](https://doc.jdb-labs.com/jlp/current/src/main/groovy/com/jdblabs/jlp/JLPPegParser.groovy.html)
A [PEG parser] implemented using the [parboiled] library. This is the A [PEG parser](http://en.wikipedia.org/wiki/Parsing_expression_grammar)
default source code parser. It is able to parse JLP documentation but leaves implemented using the [parboiled](http://www.parboiled.org) library. This
code unparsed. It can be parameterized to fit the differing documentation is the default source code parser. It is able to parse JLP documentation
styles of source languages. but leaves code unparsed. It can be parameterized to fit the differing
documentation styles of source languages.
[PEG parser]: http://en.wikipedia.org/wiki/Parsing_expression_grammar
[parboiled]: http://www.parboiled.org
### Abstract Syntax Tree ### Abstract Syntax Tree
* [SourceFile](jlp://jlp.jdb-labs.com/ast/SourceFile) * [SourceFile](https://doc.jdb-labs.com/jlp/current/src/main/groovy/com/jdblabs/jlp/JLPPegParserSourceFile.groovy.html)
The top-level AST element. This represents a source file. The top-level AST element. This represents a source file.

26
build.gradle Normal file
View File

@ -0,0 +1,26 @@
apply plugin: "groovy"
apply plugin: "maven"
group = "com.jdblabs"
version = "1.10"
repositories {
mavenLocal()
mavenCentral() }
dependencies {
compile 'org.codehaus.groovy:groovy-all:[2.4.0,)'
compile 'org.pegdown:pegdown:[1.6.0,)'
compile 'org.slf4j:slf4j-api:[1.7.10,)'
compile 'ch.qos.logback:logback-core:[1.1.2,)'
compile 'ch.qos.logback:logback-classic:[1.1.2,)'
compile 'commons-cli:commons-cli:[1.2,)'
compile 'org.apache.commons:commons-lang3:[3.3.2,)'
compile 'com.jdbernard:jdb-util:[4.0,)'
}
jar {
manifest {
attributes("Main-Class": "com.jdblabs.jlp.JLPMain")
}
}

View File

@ -1,20 +0,0 @@
<project name="Jonathan's Literate Programming" basedir="." default="release">
<import file="jdb-build-1.9.xml"/>
<property environment="env"/>
<property file="project.properties"/>
<target name="release" depends="build">
<mkdir dir="${release.dir}/lib"/>
<copy file="${build.dir}/${name}-${version}.${build.number}.jar"
tofile="${release.dir}/${name}-${version}.jar"/>
<copy todir="${release.dir}">
<fileset dir="${resources.dir}/release"/>
</copy>
<copy todir="${release.dir}/lib">
<fileset dir="${build.dir}/lib/runtime/jar"/>
</copy>
</target>
</project>

View File

@ -1,2 +0,0 @@
Implement working internal links using ``jlp://`` schema.
=========================================================

View File

@ -1 +0,0 @@
Implement language-specific code parsing and comprehension.

View File

@ -1 +0,0 @@
Support multi-line block comments.

View File

@ -1 +0,0 @@
Make the parser configurable with respect to comment delimiters.

View File

@ -1,16 +0,0 @@
Fix delimited doc block behavior.
=================================
Delimited doc blocks require that the start token be the first non-space token
on the line it is on and that the end token be on it's own line. This is not in
line with the general nature of delimited comment blocks, which do not place
any restrictions on what comes before the start delimiter or after the end
delimiter.
----
========= ===================
Created : 2011-09-07
Resolved: 2011-12-25T23:26:07
========= ===================

View File

@ -1,25 +0,0 @@
Internal links are not smart enough about cross-file linking.
=============================================================
There are two main problems with internal linking as it works now:
1. The links are resolved relative to the directory of the file being processed
when they should be resolved relative to the root output directory.
For example, consider ``@org id1`` defined in ``dir/file1.src``, then
referenced in ``dir/file2.src`` in this manner: ``[link](jlp://id1``. The
url written in ``dir/file2.html`` is ``dir/file1.html#id1``. However, when
the page is viewed, this url is interpreted relative to the directory of the
current page. So if the docs live at ``file:///docs`` then the url will
resolve to ``file:///docs/dir/dir/file1.src#id1`` instead of
``file:///docs/dir/file1.html#id1``.
2. The links do substitute the ``html`` suffix to the file names in place of the
files' suffixes. In the above example note that the actual urls end in
``.src`` like the original input files, but the expected url ends in
``.html``.
========= ==========
Created: 2011-09-08
Resolved: 2011-09-09
========= ==========

View File

@ -1,12 +0,0 @@
Encode Documentation and Code Characters for HTML
=================================================
The text of the documentation and the code is not being HTML encoded,
so some characters (most notably `<`) are causing wierd display issues
in the resulting output.
----
======== ===================
Created: 2011-12-26T00:43:44
======== ===================

View File

@ -1,12 +0,0 @@
`Include` Directive.
====================
Add a new directive: `include`. This directive allows the author to include
the contents of a URL inline in the documentation (can be an internal JLP
reference URL).
----
======== ===================
Created: 2011-12-29T11:23:52
======== ===================

View File

@ -1,12 +0,0 @@
Automatically create document `org` directives.
===============================================
The author should not have to create an `org` directive at the top of every file.
This should be done for him during parsing. Ideally the system would be smart enough
to notice if he has defined an `org` at the top of a file and use that if he has.
----
======== ===================
Created: 2012-01-04T17:12:59
======== ===================

View File

@ -1,12 +0,0 @@
Add a `title` directive for titling documents.
==============================================
Return to defaulting a title for each document. Instead of the internal document id use
something more reasonable, like the filename without the path and without the extension.
A new attribute `title` could allow the author to override this value.
----
======== ===================
Created: 2012-01-04T17:14:26
======== ===================

View File

@ -1,18 +0,0 @@
Modify `org` behavior to include simple anchors.
================================================
Currently JLP supports at most one `org` directive per block which identifies the block.
It would be useful to support multiple `org` directives within a block, particularly
when there is a large block that may have many interesting internal targets. Maybe the
`org` directive should be handled differently when it is used multiple times within a
block. This would be discovered in the generator parse phase and we could change the
LinkAnchor type at that time. During the parse phase we emit the new type of anchors
as `<a id="link-name"/>` into the document. We would also change our search for block
ids to inly look for the single-occurance type of `orgs`.
----
========= ===================
Created : 2012-01-05T11:40:35
Resolved: 2012-01-06T14:32:46
========= ===================

32
doc/jlp-notes Normal file
View File

@ -0,0 +1,32 @@
Implement special-purpose Markdown processor. This way it can be intelligent
about connecting seperated doc-blocks that should be a unit, can process thing
like link references on a whole-document basis, and can be more intelligent
about indentation, line numbers for doc blocks.
Consider building up common, generic AST nodes for code parsers so that most of
the work of integrating a new language is already done in the common code,
leaving only the very specific behavior to be implemented. For starters:
CodeUnit
: Supports things like classes in OOP, modules, even files if that is a
common organizational unit for the language.
Callable
: Supports methods, functions, subroutines, closures, anything that takes
parameters and returns a value.
Structure
: May be a useful abstraction for pure data objects (Erlang record
definitions, C/C++ unions and structs, enums, etc.).
Make Directives completely configurable. Do not hard-code an enumeration of
possible values. Make the actual directive text just another field on the class.
This allows language-specific implementations to extend the built-in Directives
easily. It also has the benefit of allowing unknown or invalid directives
through the parser without causing the source file to fail to parse. This gives
us a better way to warn the user of unknown directives. To this end it may be
good to assemble a list of defined directives at runtime, but this would not
happen in the parser itself.
Add back the @doc directive to return to private implementation notes within a
block that has so far been marked public with an @api directive.

View File

@ -1,248 +0,0 @@
<?xml version="1.0" encoding="utf-8"?>
<project name="Jonathan Bernard Build Common"
xmlns:ivy="antlib:org.apache.ivy.ant">
<property environment="env"/>
<!--======== INIT TARGETS ========-->
<target name="-init" depends="-common-init,init"/>
<target name="-common-init">
<!-- Set default values for some key properties. Since properties are
write once, any value set before this point takes precedence. -->
<property name="versioning.file" value="project.properties"/>
<property name="src.dir" value="${basedir}/src"/>
<property name="build.dir" value="${basedir}/build"/>
<property name="lib.dir" value="${basedir}/lib"/>
<property name="resources.dir" value="${basedir}/resources"/>
<property name="splash.image" value="splash.png"/>
<!--======== PATHS ========-->
<path id="groovy.classpath">
<fileset dir="${env.GROOVY_HOME}/lib">
<include name="*.jar"/>
</fileset>
</path>
<path id="groovy.embeddable">
<fileset dir="${env.GROOVY_HOME}/embeddable">
<include name="*.jar"/>
</fileset>
</path>
<path id="compile-libs">
<fileset dir="${build.dir}/lib/compile/jar">
<include name="*.jar"/>
</fileset>
</path>
<path id="runtime-libs">
<fileset dir="${build.dir}/lib/runtime/jar">
<include name="*.jar"/>
</fileset>
</path>
</target>
<target name="-init-groovy">
<taskdef name="groovyc" classpathref="groovy.classpath"
classname="org.codehaus.groovy.ant.Groovyc"/>
<taskdef name="groovy" classpathref="groovy.classpath"
classname="org.codehaus.groovy.ant.Groovy"/>
</target>
<target name="init"/>
<target name="clean" depends="-init">
<delete dir="${build.dir}"/>
</target>
<!--======== LIBRARY TARGETS ========-->
<target name="-lib" depends="-lib-local,-lib-ivy,lib"/>
<target name="lib"/>
<target name="-init-ivy">
<ivy:settings id="ivy.settings" file="ivysettings.xml"/>
</target>
<target name="-lib-ivy" depends="-init-ivy" unless="${lib.local}">
<ivy:retrieve settingsRef="ivy.settings"
pattern="${lib.dir}/[conf]/[type]/[artifact]-[revision].[ext]"
conf="compile,runtime"/>
</target>
<target name="-lib-groovy" if="${lib.local}">
<copy todir="${build.dir}/lib/runtime/jar">
<fileset dir="${env.GROOVY_HOME}/embeddable"/>
</copy>
</target>
<target name="-lib-local" if="${lib.local}">
<echo message="Resolving libraries locally."/>
<mkdir dir="${build.dir}/lib/compile/jar"/>
<mkdir dir="${build.dir}/lib/runtime/jar"/>
<copy todir="${build.dir}/lib/compile/jar" failonerror="false">
<fileset dir="${lib.dir}/compile/jar"/>
</copy>
<copy todir="${build.dir}/lib/runtime/jar" failonerror="false">
<fileset dir="${lib.dir}/runtime/jar"/>
</copy>
</target>
<!--======== VERSIONING TARGETS ========-->
<target name="increment-build-number" depends="-init">
<propertyfile file="${versioning.file}">
<entry key="build.number" default="0" type="int" value="1"
operation="+"/>
</propertyfile>
</target>
<target name="set-version" depends="-init">
<input
message="The current version is ${version}. Enter a new version: "
addproperty="new-version"/>
<propertyfile file="${versioning.file}">
<entry key="version" value="${new-version}" operation="="
type="string"/>
<entry key="build.number" value="0" type="int" operation="="/>
</propertyfile>
</target>
<!--======== COMPILATION TARGETS ========-->
<target name="-compile-groovy" depends="-init,-init-groovy,-lib,-lib-groovy">
<mkdir dir="${build.dir}/main/classes"/>
<groovyc srcdir="${src.dir}/main" destdir="${build.dir}/main/classes"
includeAntRuntime="false">
<classpath>
<path refid="groovy.classpath"/>
<path refid="compile-libs"/>
</classpath>
<javac/>
</groovyc>
</target>
<target name="-compile-java" depends="-init,-lib">
<mkdir dir="${build.dir}/main/classes"/>
<javac srcdir="${src.dir}/main" destdir="${build.dir}/main/classes"
includeAntRuntime="false" classpathref="compile-libs"/>
</target>
<target name="compile" depends="-compile-groovy"/>
<!--======== JUNIT TARGETS ========-->
<target name="-compile-tests-groovy" depends="-init,compile">
<mkdir dir="${build.dir}/test/classes"/>
<groovyc srcdir="${src.dir}/test" destdir="${build.dir}/test/classes"
includeAntRuntime="false">
<classpath>
<path refid="groovy.classpath"/>
<path refid="compile-libs"/>
<path location="${build.dir}/main/classes"/>
</classpath>
</groovyc>
</target>
<target name="-compile-tests-java" depends="-init,compile">
<mkdir dir="${build.dir}/test/classes"/>
<javac srcdir="${src.dir}/test" destdir="${build.dir}/test/classes"
includeAntRuntime="false">
<classpath>
<path refid="compile-libs"/>
<path location="${build.dir}/main/classes"/>
</classpath>
</javac>
</target>
<target name="compile-tests" depends="-compile-tests-groovy"/>
<target name="run-tests" depends="compile-tests,resources-test">
<junit printsummary="true">
<classpath>
<path refid="groovy.classpath"/>
<path refid="compile-libs"/>
<path location="${build.dir}/main/classes"/>
<path location="${build.dir}/test/classes"/>
</classpath>
<formatter type="plain" usefile="false"/>
<batchtest>
<fileset dir="${build.dir}/test/classes">
<include name="**/*"/>
</fileset>
</batchtest>
</junit>
</target>
<!--======== RESOURCES TARGETS ========-->
<target name="resources" depends="-init">
<mkdir dir="${build.dir}/main/classes"/>
<copy todir="${build.dir}/main/classes" failonerror="false">
<fileset dir="${resources.dir}/main/"/>
</copy>
</target>
<target name="resources-test" depends="-init">
<mkdir dir="${build.dir}/test/classes"/>
<copy todir="${build.dir}/test/classes" failonerror="false">
<fileset dir="${resources.dir}/test/"/>
</copy>
</target>
<!--======== BUILD TARGETS ========-->
<target name="-build-modular-lib" unless="executable.jar"
depends="compile,increment-build-number,resources">
<jar destfile="${build.dir}/${name}-${version}.${build.number}.jar"
basedir="${build.dir}/main/classes"/>
</target>
<target name="-build-modular-executable" if="executable.jar"
depends="compile,increment-build-number,resources">
<pathconvert property="jar.classpath" pathsep=" " refid="runtime-libs">
<mapper>
<chainedmapper>
<!-- remove absolute path -->
<flattenmapper />
<!-- add lib/ prefix -->
<globmapper from="*" to="lib/*" />
</chainedmapper>
</mapper>
</pathconvert>
<jar destfile="${build.dir}/${name}-${version}.${build.number}.jar"
basedir="${build.dir}/main/classes">
<manifest>
<attribute name="Main-Class" value="${main.class}"/>
<attribute name="Class-Path" value="${jar.classpath}"/>
<attribute name="SplashScreen-Image" value="${splash.image}"/>
</manifest>
</jar>
</target>
<target name="-build-modular"
depends="-build-modular-lib,-build-modular-executable"/>
<target name="-build-packed-libs"
depends="compile,increment-build-number,resources">
<unjar destdir="${build.dir}/main/classes">
<fileset dir="${build.dir}/lib/runtime/jar"/>
</unjar>
<jar destfile="${build.dir}/${name}-${version}.${build.number}.jar"
basedir="${build.dir}/main/classes"/>
</target>
<target name="build" depends="-build-modular"/>
</project>

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@ -1,8 +0,0 @@
#Fri, 27 Jan 2012 18:21:58 -0600
name=jlp
version=1.7
build.number=24
lib.local=true
release.dir=release
main.class=com.jdblabs.jlp.JLPMain
executable.jar=true

View File

@ -18,7 +18,7 @@ public abstract class JLPBaseGenerator {
/** /**
* The generator works in close conjunction with a JLP Processor. * The generator works in close conjunction with a JLP Processor.
* This tight coupling in intended for these two classes. The distiction * This tight coupling is intended for these two classes. The distiction
* between the two classes is scope. The Processor class is concerned with * between the two classes is scope. The Processor class is concerned with
* data and behavior common to the whole documentation process whereas the * data and behavior common to the whole documentation process whereas the
* Generator is concerned only with one file. The Generator does have * Generator is concerned only with one file. The Generator does have

View File

@ -19,7 +19,7 @@ import org.slf4j.LoggerFactory
*/ */
public class JLPMain { public class JLPMain {
public static final String VERSION = "1.7" public static final String VERSION = "1.10"
private static Logger log = LoggerFactory.getLogger(JLPMain.class) private static Logger log = LoggerFactory.getLogger(JLPMain.class)
@ -37,19 +37,19 @@ public class JLPMain {
/// : Print help information. /// : Print help information.
cli.h('Print this help information.', longOpt: 'help', required: false) cli.h('Print this help information.', longOpt: 'help', required: false)
/// -o, --outputdir /// -o, --outputdir <output-directory>
/// : Set the output directory where the documentation will be /// : Set the output directory where the documentation will be
/// written. /// written.
cli.o("Output directory (defaults to 'jlp-docs').", cli.o("Output directory (defaults to 'jlp-docs').",
longOpt: 'output-dir', args: 1, argName: "output-dir", longOpt: 'output-dir', args: 1, argName: "output-dir",
required: false) required: false)
/// --css-file /// --css-file <file>
/// : Specify an alternate CSS file for the output documentation. /// : Specify an alternate CSS file for the output documentation.
cli._('Use <css-file> for the documentation css.', cli._('Use <css-file> for the documentation css.',
longOpt: 'css-file', args: 1, required: false, argName: 'css-file') longOpt: 'css-file', args: 1, required: false, argName: 'css-file')
/// --relative-path-root /// --relative-path-root <root-directory>
/// : Override the current working directory. This is useful if you /// : Override the current working directory. This is useful if you
/// are invoking jlp remotely, or if the current working directory /// are invoking jlp remotely, or if the current working directory
/// is incorrectly set by the executing environment. /// is incorrectly set by the executing environment.
@ -113,7 +113,7 @@ public class JLPMain {
/// Resolve the file against our relative root. /// Resolve the file against our relative root.
if (!cssFile.isAbsolute()) { if (!cssFile.isAbsolute()) {
cssFile = new File(pathRoot, cssFile.path) } cssFile = new File(pathRoot, cssFile.path) }
/// Finally, make sure the CSS file actually exists. /// Finally, make sure the CSS file actually exists.
if (cssFile.exists()) { if (cssFile.exists()) {
css = cssFile css = cssFile
@ -142,14 +142,14 @@ public class JLPMain {
/// For each filename we try to resolve it to an actual file /// For each filename we try to resolve it to an actual file
/// relative to our root. /// relative to our root.
File file = new File(filename) File file = new File(filename)
if (!file.isAbsolute()) { file = new File(pathRoot, filename) } if (!file.isAbsolute()) { file = new File(pathRoot, filename) }
/// If this file does not exist, warn the user and skip it. /// If this file does not exist, warn the user and skip it.
if (!file.exists()) { if (!file.exists()) {
System.err.println( System.err.println(
"'${file.canonicalPath}' does not exist: ignored.") "'${file.canonicalPath}' does not exist: ignored.")
return } return }
/// If this file is a directory, we want to add all the files in it /// If this file is a directory, we want to add all the files in it
/// to our input list, recursing into all the subdirectories and /// to our input list, recursing into all the subdirectories and
/// adding their files as well. We will ignore hidden files. /// adding their files as well. We will ignore hidden files.

View File

@ -13,7 +13,7 @@ import com.jdblabs.jlp.ast.SourceFile;
* be an abstract class implementing methods that take additional input for * be an abstract class implementing methods that take additional input for
* convenience. * convenience.
* *
* [`SourceFile`]: jlp://jlp.jdb-labs.com/SourceFile * [`SourceFile`]: jlp://jlp.jdb-labs.com/ast/SourceFile
* *
* @org jlp.jdb-labs.com/JLPParser * @org jlp.jdb-labs.com/JLPParser
*/ */

View File

@ -40,8 +40,8 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
*/ */
/** /**
* #### Full constructor * #### Full constructor.
* This allows the caller to specific all four of the comment delimiters * This allows the caller to specify all four of the comment delimiters
* recognized by the parser. * recognized by the parser.
*/ */
public JLPPegParser(String mdocStart, String mdocEnd, public JLPPegParser(String mdocStart, String mdocEnd,
@ -52,6 +52,20 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
SDOC_START = String(sdocStart).label("SDOC_START"); SDOC_START = String(sdocStart).label("SDOC_START");
MDOC_LINE_START = AnyOf(mdocLineStart).label("MDOC_LINE_START"); } MDOC_LINE_START = AnyOf(mdocLineStart).label("MDOC_LINE_START"); }
/**
* #### Full constructor supporting multiple comment types.
* This allows the caller to specify all four of the comment delimiters
* recognized by the parser, supplying multiple types of comment
* delimiters.
*/
public JLPPegParser(List mdocStarts, List mdocEnds,
String mdocLineStarts, List sdocStart) {
MDOC_START = FirstOf(mdocStarts.toArray()).label("MDOC_START");
MDOC_END = FirstOf(mdocEnds.toArray()).label("MDOC_END");
SDOC_START = FirstOf(mdocStarts.toArray()).label("SDOC_START");
MDOC_LINE_START = AnyOf(mdocLineStarts).label("MDOC_LINE_START"); }
/** /**
* #### Single-line comments only constructor. * #### Single-line comments only constructor.
* This allows the caller to easily set up the parser to only recognize * This allows the caller to easily set up the parser to only recognize
@ -110,7 +124,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
public Rule SourceFile() { public Rule SourceFile() {
return Sequence( return Sequence(
/// At the start of processing a new SourceFile, we need to set up /// At the start of processing a new SourceFile, we need to set up
/// our internal state. /// our internal state.
/// Clear the line count. /// Clear the line count.
clearLineCount(), clearLineCount(),
@ -125,7 +139,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
FirstOf( FirstOf(
/// Match a whole Block. This pushes a Block on the stack. /// Match a whole Block. This pushes a Block on the stack.
Block(), Block(),
/// A standalone DocBlock. We will create an empty CodeBlock /// A standalone DocBlock. We will create an empty CodeBlock
/// to pair with it, then push a new Block onto the stack /// to pair with it, then push a new Block onto the stack
@ -204,7 +218,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
Rule DocBlock() { return FirstOf(SDocBlock(), MDocBlock()); } Rule DocBlock() { return FirstOf(SDocBlock(), MDocBlock()); }
/** /**
* #### SDocBlock * #### SDocBlock
* Parses the rule: * Parses the rule:
* *
* SDocBlock = (SDirective / SDocText)+ * SDocBlock = (SDirective / SDocText)+
@ -221,7 +235,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
addToDocBlock((ASTNode) pop())))); } addToDocBlock((ASTNode) pop())))); }
/** /**
* #### MDocBlock * #### MDocBlock
* Parses the rule: * Parses the rule:
* *
* MDocBlock = MDOC_START (MDirective / MDocText)+ MDOC_END * MDocBlock = MDOC_START (MDirective / MDocText)+ MDOC_END
@ -243,7 +257,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
addToDocBlock((ASTNode) pop()))), addToDocBlock((ASTNode) pop()))),
MDOC_END); } MDOC_END); }
/** /**
* #### CodeBlock * #### CodeBlock
* Parses the rule: * Parses the rule:
* *
* CodeBlock = (RemainingCodeLine)+ * CodeBlock = (RemainingCodeLine)+
@ -259,7 +273,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
addToCodeBlock(match())))); } addToCodeBlock(match())))); }
/** /**
* #### SDirective * #### SDirective
* Parses the rule: * Parses the rule:
* *
* SDirective = SDocLineStart AT (SLongDirective / SShortDirective) * SDirective = SDocLineStart AT (SLongDirective / SShortDirective)
@ -273,7 +287,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
SDocLineStart(), AT, FirstOf(SLongDirective(), SShortDirective())); } SDocLineStart(), AT, FirstOf(SLongDirective(), SShortDirective())); }
/** /**
* #### MDirective * #### MDirective
* Parses the rule: * Parses the rule:
* *
* MDirective = MDocLineStart? AT (MLongDirective / MShortDirective) * MDirective = MDocLineStart? AT (MLongDirective / MShortDirective)
@ -288,11 +302,11 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
AT, FirstOf(MLongDirective(), MShortDirective())); } AT, FirstOf(MLongDirective(), MShortDirective())); }
/** /**
* #### SLongDirective * #### SLongDirective
* Parses the rule: * Parses the rule:
* *
* SLongDirective = * SLongDirective =
* (API_DIR / EXAMPLE_DIR) RemainingSDocLine SDocText? * (API_DIR / EXAMPLE_DIR / PARAM_DIR) RemainingSDocLine SDocText?
* *
* Pushes a [`Directive`] node onto the stack. * Pushes a [`Directive`] node onto the stack.
* *
@ -302,23 +316,23 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
return Sequence( return Sequence(
push(curLineNum), push(curLineNum),
FirstOf(API_DIR, EXAMPLE_DIR), push(match()), FirstOf(API_DIR, EXAMPLE_DIR, PARAM_DIR), push(match()),
RemainingSDocLine(), push(match()), RemainingSDocLine(), push(match()),
Optional(Sequence( Optional(Sequence(
SDocText(), SDocText(),
swap(), swap(),
push(popAsString() + ((DocText) pop()).value))), push(popAsString() + ((DocText) pop()).value))),
push(new Directive(popAsString(), popAsString(), popAsInt(), push(new Directive(popAsString(), popAsString(), popAsInt(),
(DocBlock)peek()))); } (DocBlock)peek()))); }
/** /**
* #### MLongDirective * #### MLongDirective
* Parses the rule: * Parses the rule:
* *
* MLongDirective = * MLongDirective =
* (API_DIR / EXAMPLE_DIR) RemainingMDocLine MDocText? * (API_DIR / EXAMPLE_DIR / PARAM_DIR) RemainingMDocLine MDocText?
* *
* Pushes a [`Directive`] node onto the stack. * Pushes a [`Directive`] node onto the stack.
* *
@ -328,8 +342,8 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
return Sequence( return Sequence(
push(curLineNum), push(curLineNum),
FirstOf(API_DIR, EXAMPLE_DIR), push(match()), FirstOf(API_DIR, EXAMPLE_DIR, PARAM_DIR), push(match()),
RemainingMDocLine(), push(match()), RemainingMDocLine(), push(match()),
Optional(Sequence( Optional(Sequence(
MDocText(), MDocText(),
@ -340,7 +354,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
(DocBlock) peek()))); } (DocBlock) peek()))); }
/** /**
* #### SShortDirective * #### SShortDirective
* Parses the rule: * Parses the rule:
* *
* SShortDirective = * SShortDirective =
@ -356,12 +370,12 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
push(curLineNum), push(curLineNum),
FirstOf(AUTHOR_DIR, ORG_DIR, INCLUDE_DIR, COPYRIGHT_DIR), push(match()), FirstOf(AUTHOR_DIR, ORG_DIR, INCLUDE_DIR, COPYRIGHT_DIR), push(match()),
RemainingSDocLine(), RemainingSDocLine(),
push(new Directive(match().trim(), popAsString(), popAsInt(), push(new Directive(match().trim(), popAsString(), popAsInt(),
(DocBlock) peek()))); } (DocBlock) peek()))); }
/** /**
* #### MShortDirective * #### MShortDirective
* Parses the rule: * Parses the rule:
* *
* MShortDirective = * MShortDirective =
@ -382,7 +396,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
(DocBlock) peek()))); } (DocBlock) peek()))); }
/** /**
* #### SDocText * #### SDocText
* Parses the rule: * Parses the rule:
* *
* SDocText = (SDocLineStart !AT RemainingSDocLine)+ * SDocText = (SDocLineStart !AT RemainingSDocLine)+
@ -399,7 +413,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
addToDocText(match())))); } addToDocText(match())))); }
/** /**
* #### MDocText * #### MDocText
* Parses the rule: * Parses the rule:
* *
* MDocText = (MDocLineStart? !AT RemainingMDocLine)+ * MDocText = (MDocLineStart? !AT RemainingMDocLine)+
@ -417,7 +431,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
addToDocText(match())))); } addToDocText(match())))); }
/** /**
* #### SDocLineStart * #### SDocLineStart
* Parses the rule: * Parses the rule:
* *
* SDocLineStart = SPACE* SDOC_START SPACE? * SDocLineStart = SPACE* SDOC_START SPACE?
@ -427,7 +441,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
ZeroOrMore(SPACE), SDOC_START, Optional(SPACE)); } ZeroOrMore(SPACE), SDOC_START, Optional(SPACE)); }
/** /**
* #### MDocLineStart * #### MDocLineStart
* Parses the rule: * Parses the rule:
* *
* MDocLineStart = SPACE* !MDOC_END MDOC_LINE_START SPACE? * MDocLineStart = SPACE* !MDOC_END MDOC_LINE_START SPACE?
@ -437,7 +451,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
ZeroOrMore(SPACE), TestNot(MDOC_END), MDOC_LINE_START, Optional(SPACE)); } ZeroOrMore(SPACE), TestNot(MDOC_END), MDOC_LINE_START, Optional(SPACE)); }
/** /**
* #### RemainingSDocLine * #### RemainingSDocLine
* Parses the rule: * Parses the rule:
* *
* RemainingSDocLine = ((!EOL)* EOL) / ((!EOL)+ EOI) * RemainingSDocLine = ((!EOL)* EOL) / ((!EOL)+ EOI)
@ -448,10 +462,10 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
Sequence(OneOrMore(NOT_EOL), EOI, incLineCount())); } Sequence(OneOrMore(NOT_EOL), EOI, incLineCount())); }
/** /**
* #### RemainingMDocLine * #### RemainingMDocLine
* Parses the rule: * Parses the rule:
* *
* RemainingMDocLine = * RemainingMDocLine =
* ((!(EOL / MDOC_END))* EOL) / * ((!(EOL / MDOC_END))* EOL) /
* ((!MDOC_END)+) * ((!MDOC_END)+)
*/ */
@ -467,10 +481,10 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
OneOrMore(Sequence(TestNot(MDOC_END), ANY))); } OneOrMore(Sequence(TestNot(MDOC_END), ANY))); }
/** /**
* #### RemainingCodeLine * #### RemainingCodeLine
* Parses the rule: * Parses the rule:
* *
* RemainingCodeLine = * RemainingCodeLine =
* ((!(EOL / MDOC_START / SDocLineStart))* EOL) / * ((!(EOL / MDOC_START / SDocLineStart))* EOL) /
* (!(MDOC_START / SDocLineStart))+ * (!(MDOC_START / SDocLineStart))+
*/ */
@ -497,6 +511,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
Rule EXAMPLE_DIR = IgnoreCase("example"); Rule EXAMPLE_DIR = IgnoreCase("example");
Rule INCLUDE_DIR = IgnoreCase("include"); Rule INCLUDE_DIR = IgnoreCase("include");
Rule ORG_DIR = IgnoreCase("org"); Rule ORG_DIR = IgnoreCase("org");
Rule PARAM_DIR = IgnoreCase("param");
/// #### Hard-coded terminals. /// #### Hard-coded terminals.
Rule AT = Ch('@').label("AT"); Rule AT = Ch('@').label("AT");
@ -513,8 +528,8 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
/// ### Utility/Helper Functions. ### /// ### Utility/Helper Functions. ###
/// --------------------------------- /// ---------------------------------
/// The `popAs` functions exist primarily to make the parser rules more readable /// The `popAs` functions exist primarily to make the parser rules more
/// by providing shortcuts for common casts. /// readable by providing shortcuts for common casts.
String popAsString() { return (String) pop(); } String popAsString() { return (String) pop(); }
Integer popAsInt() { return (Integer) pop(); } Integer popAsInt() { return (Integer) pop(); }
@ -524,8 +539,8 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
/** /**
* #### addToDocBlock * #### addToDocBlock
* Add the given block to the [`SourceFile`] node expected to be at the top of * Add the given block to the [`SourceFile`] node expected to be at the top
* the parser value stack. * of the parser value stack.
* *
* [`SourceFile`]: jlp://jlp.jdb-labs.com/ast/SourceFile * [`SourceFile`]: jlp://jlp.jdb-labs.com/ast/SourceFile
*/ */
@ -535,8 +550,8 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
/** /**
* #### addToDocBlock * #### addToDocBlock
* Add the given [`Directive`] or [`DocText`] to the [`DocBlock`] expected to be at the * Add the given [`Directive`] or [`DocText`] to the [`DocBlock`] expected
* top of the parser value stack. * to be at the top of the parser value stack.
* *
* [`Directive`]: jlp://jlp.jdb-labs.com/ast/Directive * [`Directive`]: jlp://jlp.jdb-labs.com/ast/Directive
* [`DocText`]: jlp://jlp.jdb-labs.com/ast/DocText * [`DocText`]: jlp://jlp.jdb-labs.com/ast/DocText
@ -550,7 +565,7 @@ public class JLPPegParser extends BaseParser<Object> implements JLPParser {
docBlock.docTexts.add((DocText) an); } docBlock.docTexts.add((DocText) an); }
else { throw new IllegalStateException(); } else { throw new IllegalStateException(); }
return push(docBlock); } return push(docBlock); }
boolean addToCodeBlock(String line) { boolean addToCodeBlock(String line) {
CodeBlock codeBlock = (CodeBlock) pop(); CodeBlock codeBlock = (CodeBlock) pop();
codeBlock.lines.put(curLineNum - 1, line); codeBlock.lines.put(curLineNum - 1, line);

View File

@ -217,7 +217,7 @@ public class LiterateMarkdownGenerator extends JLPBaseGenerator {
/// Close the table row. /// Close the table row.
sb.append('</tr>') } sb.append('</tr>') }
/** @api Emit a [`DocBlock`](jlp://jlp/jdb-labs.com/ast/DocBlock). */ /** @api Emit a [`DocBlock`](jlp://jlp.jdb-labs.com/ast/DocBlock). */
protected String emit(DocBlock docBlock) { protected String emit(DocBlock docBlock) {
/// Create a queue for the doc block elements, we are going to /// Create a queue for the doc block elements, we are going to
/// sort them by type and line number /// sort them by type and line number
@ -248,7 +248,7 @@ public class LiterateMarkdownGenerator extends JLPBaseGenerator {
[lineNumber, line] } [lineNumber, line] }
/// Sort by line number. /// Sort by line number.
codeLines.sort({ i1, i2 -> i1[0] - i2[0] } as Comparator) codeLines.sort { i1, i2 -> i1[0] <=> i2[0] }
codeLines = codeLines.collect { arr -> arr[1] } codeLines = codeLines.collect { arr -> arr[1] }
@ -281,6 +281,9 @@ public class LiterateMarkdownGenerator extends JLPBaseGenerator {
case DirectiveType.Example: case DirectiveType.Example:
return directive.value return directive.value
case DirectiveType.Param:
return "" // TODO: can we do better here, even though we're
// not understanding the source yet?
// TODO: // TODO:
// case DirectiveType.Include: // case DirectiveType.Include:

View File

@ -125,15 +125,13 @@ public class Processor {
/// is more than one file with the same name we will include the /// is more than one file with the same name we will include the
/// file's parent directory as well. /// file's parent directory as well.
inputFiles.each { file -> inputFiles.each { file ->
// Get the relative path as path elements. // Get the relative path as path elements.
def relPath = getRelativeFilepath(inputRoot, file) def relPath = getRelativeFilepath(inputRoot, file)
def pathParts = relPath.split('/') as List def pathParts = relPath.split('/|\\\\') as List
// Get our file type.
def fileType = sourceTypeForFile(file)
// We will skip binary files and files we know nothing about. // We will skip binary files and files we know nothing about.
def fileType = sourceTypeForFile(file)
if (fileType == 'binary' || fileType == 'unknown') { return; } if (fileType == 'binary' || fileType == 'unknown') { return; }
// Start with just the file name. // Start with just the file name.
@ -162,7 +160,7 @@ public class Processor {
// TODO: better error detection and handling // TODO: better error detection and handling
currentDoc.sourceAST = parser.parse(currentDoc.sourceFile.text) currentDoc.sourceAST = parser.parse(currentDoc.sourceFile.text)
if (currentDoc.sourceAST == null) { if (currentDoc.sourceAST == null) {
log.warn("Unable to parse '{}'. Ignoring this document.", currentDocId) log.warn("Unable to parse '{}'. Ignoring this document.", currentDocId)
badDocs << currentDocId }} badDocs << currentDocId }}
@ -188,7 +186,7 @@ public class Processor {
/// * Write the output to the output directory. /// * Write the output to the output directory.
processDocs { processDocs {
/// Create the path and file object for the output file /// Create the path and file object for the output file
String relativePath = String relativePath =
getRelativeFilepath(inputRoot, currentDoc.sourceFile) getRelativeFilepath(inputRoot, currentDoc.sourceFile)
@ -240,7 +238,7 @@ public class Processor {
* : Return the link as-is. * : Return the link as-is.
* *
* *absolute path (starts with `/`)* * *absolute path (starts with `/`)*
* : Returns the link resolved against the output root. * : Returns the link resolved against the output root.
* *
* *relative path (no leading `/`)* * *relative path (no leading `/`)*
* : Returns the link resolved against the `TargetDoc` passed in. * : Returns the link resolved against the `TargetDoc` passed in.
@ -283,12 +281,12 @@ public class Processor {
case ~/^\w+:.*/: return link case ~/^\w+:.*/: return link
/// Absolute link, resolve relative to the output root. /// Absolute link, resolve relative to the output root.
case ~/^\/.*/: case ~/^\/.*/:
/// Our link should be the relative path (if needed) plus the /// Our link should be the relative path (if needed) plus the
/// link without the leading `/`. /// link without the leading `/`.
def relPath = getRelativeFilepath(targetDoc.sourceFile, inputRoot) def relPath = getRelativeFilepath(targetDoc.sourceFile, inputRoot)
return (relPath ? "${relPath}/" : "") + link[1..-1] return (relPath ? "${relPath}/" : "") + link[1..-1]
/// Relative link, resolve using the output root and the source /// Relative link, resolve using the output root and the source
/// document relative to the input root. /// document relative to the input root.
default: default:
@ -297,7 +295,7 @@ public class Processor {
/** /**
* #### getRelativeFilepath * #### getRelativeFilepath
* Assuming our current directory is `root`, get the relative path to * Assuming our current directory is `root`, get the relative path to
* `file`. * `file`.
* @org jlp.jdb-labs.com/Processor/getRelativeFilepath * @org jlp.jdb-labs.com/Processor/getRelativeFilepath
*/ */
@ -306,8 +304,8 @@ public class Processor {
if (!root.isDirectory()) root= root.parentFile if (!root.isDirectory()) root= root.parentFile
/// Split both paths into their individual parts. /// Split both paths into their individual parts.
def rootPath = root.canonicalPath.split('/') def rootPath = root.canonicalPath.split('/|\\\\')
def filePath = file.canonicalPath.split('/') def filePath = file.canonicalPath.split('/|\\\\')
def relativePath = [] def relativePath = []
@ -331,17 +329,17 @@ public class Processor {
* #### getCommonParent * #### getCommonParent
* Find the common parent directory to the given files. * Find the common parent directory to the given files.
* @org jlp.jdb-labs.com/Processor/getCommonParent * @org jlp.jdb-labs.com/Processor/getCommonParent
*/ */
public static File getCommonParent(File file1, File file2) { public static File getCommonParent(File file1, File file2) {
def path1 = file1.canonicalPath.split('/') def path1 = file1.canonicalPath.split('/|\\\\')
def path2 = file2.canonicalPath.split('/') def path2 = file2.canonicalPath.split('/|\\\\')
def newPath = [] def newPath = []
// build new commonPath based on matching paths so far // build new commonPath based on matching paths so far
int i = 0 int i = 0
while (i < Math.min(path1.length, path2.length) && while (i < Math.min(path1.length, path2.length) &&
path1[i] == path2[i]) { path1[i] == path2[i]) {
newPath << path2[i] newPath << path2[i]
i++ } i++ }
@ -440,8 +438,8 @@ public class Processor {
break break
case 'html': case 'xml': case 'html': case 'xml':
parsers[sourceType] = Parboiled.createParser( parsers[sourceType] = Parboiled.createParser(
JLPPegParser, '<!--', '-->', JLPPegParser, '<!--!', '-->',
'#$%^&*()_-+=|;:\'",<>?~`', '<<?') '!#$%^&*()_-+=|;:\'",<>?~`', '<<?')
break break
case 'sql': case 'sql':
parsers[sourceType] = Parboiled.createParser(JLPPegParser, parsers[sourceType] = Parboiled.createParser(JLPPegParser,

View File

@ -50,10 +50,15 @@ public class Directive extends ASTNode {
* the [`LinkAnchor`] documentation for more information about link * the [`LinkAnchor`] documentation for more information about link
* anchors. * anchors.
* *
* Param
* : Used to document a parameter to a function, method, or subroutine.
* This is typically used in API documentation, but no generator
* currently knows how to do anything intelligent with this directive.
*
* [`LinkAnchor`]: jlp://jlp.jdb-labs.com/LinkAnchor * [`LinkAnchor`]: jlp://jlp.jdb-labs.com/LinkAnchor
*/ */
public static enum DirectiveType { public static enum DirectiveType {
Api, Author, Copyright, Example, Include, Org; Api, Author, Copyright, Example, Include, Org, Param;
public static DirectiveType parse(String typeString) { public static DirectiveType parse(String typeString) {
valueOf(typeString.toLowerCase().capitalize()) } } valueOf(typeString.toLowerCase().capitalize()) } }