jdb/timestamper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Changed Data Model

Browse Source 
------------------

* Created the extended data table. This is a more generic version of the
  extended data field that was on ``ts_user``. Instead of arbitrary key-value
  pairs going in a list on the user record we will have an additional table, a
  one to many relationship between existing tables and the ``ts_ext_data``
  table, each row being an extended value of the corresponding record.
* Added support to the ``timestamper:create_tables/1`` and
  ``timestamper_dev:create_table/1`` functions for the new ``ts_ext_data`` table.

Documentation
-------------

* Added some general docs about the DB layer code.
* Added two new issues, *D0020*: Entry exclusion filters and *D0021*: notes width.

API Changes
-----------

Necessitated by the change to the data model.

* Updated ``ts_api`` data-retrieval functions to use extended data:

    * ``get_user_summary/2``
    * ``list_timelines/2``
    * ``list_entries/3``

* Updated ``ts_api:put_timeline/3`` to parse the extended data supplied by the
  caller. *FIXME* it is not actually saving this data.
* TODO: Started updating ``ts_api:post_entry/3`` to handle extended data, need
  to finish.

Database Layer
--------------

Changes necessitated by the change to the model.

* Added ``ts_common:do_set_ext_data/2`` which iterates through the extended data
  key-value pairs calling ``ts_ext_data:set_property/3``. It does not provide a
  transaction context.
* Split ``ts_common:new/1`` and ``ts_common:update/1`` functions into multiple
  functions to prevent code-duplication:

    * ``do_{new,update}`` contains the code that performs integrity checks and
      the actual database write function. It does this assuming that it is being
      called from within the context of an mnesia transaction (uses
      ``mnesia:read`` and ``mnesia:write``).
    * ``{new,update}/1`` performs the same function as previously. The
      implementation changed from using mnesia ``dirty_*`` calls to prociding a
      transaction and calling ``do_{new,update}/1``.
    * ``{new,update}/2`` expect the record to update/create and the extended
      data to write atomically with the record. They provide a transaction
      context, call ``do_{new,update}/1``, then call ``do_set_ext_data/2``.

* Similar to the refactoring of ``ts_common:{new,update}/1``,
  ``ts_entry:{new,update}/1`` have been refactored into multiple methods each to
  support extended data properties:

    * ``do_{new,update}/1`` perform the actual update assuming we have already
      established an mnesia transaction.
    * ``{new,update}/1`` behave the same as they used to, but now do so by
      creating an mnesia transaction and calling ``do_{new,update}/1``.
    * ``{new,update}/2`` create an mnesia transaction, call
      ``do_{new,update}/1``, and then call ``ts_common:do_set_ext_data/2``.

* Again similar to the refactoring of ``ts_common:{new,update}/1``,
  ``ts_user:{new,update}/1`` have been refactored into multiple methods each to
  support extended data properties:

    * ``do_{new,update}/1`` perform the actual update assuming we have already
      established an mnesia transaction.
    * ``{new,update}/1`` behave the same as they used to, but now do so by
      creating an mnesia transaction and calling ``do_{new,update}/1``.
    * ``{new,update}/2`` create an mnesia transaction, call
      ``do_{new,update}/1``, and then call ``ts_common:do_set_ext_data/2``.

* Created the ``ts_ext_data`` module as the interface to the extended data
  properties introduced in the data model:

    * ``create_table/1`` performs the same function as it does in the other db
      layer modules, creates the table with the appropriate structure given the
      more general table options desired (location, storage type, etc.).
    * ``set_property/3`` takes a record, a property key, and a property value as
      input and sets the property described by the property key on the record to
      the given value, assuming this is a valid property for the record to have.
      For example, currently the ``ts_user`` record can have an associated
      property, ``last_timeline``, which represents the last timeline the user
      was working with. Trying to pass this property with a ``ts_entry`` record
      would result in an exception. This function uses
      ``ts_ext_data:do_set_property/3`` as its underlying implementation.
    * ``get_property/2`` takes a record and a property key and returns the value
      of that property for the given record, or ``not_set`` if the property has
      not been set on that record. This method creates its own mnesia
      transaction.
    * ``get_properties/1`` takes a record and returns a list of key-value tuples
      representing all of the extended data properties set for the given record.
      This method creates its own mnesia transaction.
    * ``do_set_property/3`` takes a record reference (not the whole record), a
      a property key, and adds the property assignment to the ``ts_ext_data``
      table. It creates its own mnesia transaction.

* Added ``new/2`` and ``update/2`` to the ``ts_timeline`` module to support
  extended data properties. They delegate implementation to
  ``ts_common:{new,update}/2``.

JSON Encoding/Decoding
----------------------

Changes necessitated by the change to the data model.

The JSON objects now contain a potentially unlimited number of fields, as each
extended data property is encoded as a seperate field, and looks no different
from any of the required fields on the object. The intended explanation in API
documentation is that each object type (``user``, ``timeline``, or ``entry``)
now has both *required* fields that *MUST* be present in every message in either
direction and *optional* fields that may or may not be present in any
communication with the API. There should be a clear distinction between which
fields are required and which are optional. It might also be a good idea to
provide a suggested default for optional values when they are not present.

* Updated documentation about JSON record structures to reflect the fact that
  there are now potentially many optional attributes in addition to the required
  attributes for each record.
* ``record_to_ejson/1`` refactored to ``record_to_ejson/2`` which also takes
  the extended data attributes and appends them as additional attributes to
  the end of the record structure.
* Created ``ext_data_to_ejson/{1,2}`` to provide a mechanism for reformatting
  extended data properties whose internal representations are not immediately
  translatable into JSON. Currently only the ``entry_exclusions`` property,
  which is a list of strings, needs to be treated this way (changing from
  ``[val, val]`` to ``{array, [val, val]}`` as needed by ``json:encode/1``.
  ``ext_data_to_ejson/1`` acts as a more user-friendly facade to
  ``ext_data_to_ejson/2``.
* Rewrote ``ejson_to_record/{2,3}`` and ``ejson_to_record_strict/{2,3}`` to
  handle extended data. They now use a common method, ``construct_record/3`` to
  create the actual record object and extended data key-value list.
  ``ejson_to_record_strict/{2,3}`` only differs in that it checks for the
  presence of each required field of the record after the record is constructed.
  The three-parameter versions of these functions also take in the intended
  reference for the constructed record, replacing anything that is in the EJSON
  body as the record reference (useful when the body does not have the record
  ids). These methods now return a tuple: ``{Record, ExtData}`` instead of just
  the record.
* Created ``construct_record/3`` takes a record and the EJSON fields from the
  input object. The third parameter is an accumulator for the extended data
  properties found when constructing the record. This method works by iterating
  over the list of input fields. It recognizes any required fields and updates
  the record being built with the value. Any fields it does not recognize it
  assumes are extended data properties and adds to its list. When all input
  fields have been visited it returns the record and list it has constructed.
* ``ejson_to_ext_data/{1,2}`` is the inverse of ``ext_data_to_ejson/{1,2}``.
  *TODO*: this method is not actually being used by the ``ejson_to_record*``
  methods.
This commit is contained in:
Jonathan Bernard 2011-06-14 15:24:57 -05:00
parent 99d04935cb
commit 8bb50c058d
20 changed files with 431 additions and 93 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

107
.ide/vim-views/ts_common.erl.view Normal file
View File

@ -0,0 +1,107 @@
let s:so_save = &so | let s:siso_save = &siso | set so=0 siso=0
argglobal
edit ~/projects/jdb-labs/timestamper/web-app/src/ts_common.erl
setlocal keymap=
setlocal noarabic
setlocal autoindent
setlocal balloonexpr=
setlocal nobinary
setlocal bufhidden=
setlocal buflisted
setlocal buftype=
setlocal nocindent
setlocal cinkeys=0{,0},0),:,0#,!^F,o,O,e
setlocal cinoptions=
setlocal cinwords=if,else,while,do,for,switch
setlocal comments=s1:/*,mb:*,ex:*/,://,b:#,:%,:XCOMM,n:>,fb:-
setlocal commentstring=/*%s*/
setlocal complete=.,w,b,u,t,i
setlocal completefunc=
setlocal nocopyindent
setlocal nocursorcolumn
setlocal nocursorline
setlocal define=
setlocal dictionary=
setlocal nodiff
setlocal equalprg=
setlocal errorformat=
setlocal expandtab
if &filetype != 'erlang'
setlocal filetype=erlang
endif
setlocal foldcolumn=0
setlocal foldenable
setlocal foldexpr=0
setlocal foldignore=#
setlocal foldlevel=0
setlocal foldmarker={{{,}}}
setlocal foldmethod=manual
setlocal foldminlines=1
setlocal foldnestmax=20
setlocal foldtext=foldtext()
setlocal formatexpr=
setlocal formatoptions=tcq
setlocal formatlistpat=^\\s*\\d\\+[\\]:.)}\\t\ ]\\s*
setlocal grepprg=
setlocal iminsert=2
setlocal imsearch=2
setlocal include=
setlocal includeexpr=
setlocal indentexpr=
setlocal indentkeys=0{,0},:,0#,!^F,o,O,e
setlocal noinfercase
setlocal iskeyword=@,48-57,_,192-255
setlocal keywordprg=
setlocal nolinebreak
setlocal nolisp
setlocal nolist
setlocal makeprg=
setlocal matchpairs=(:),{:},[:]
setlocal nomodeline
setlocal modifiable
setlocal nrformats=octal,hex
setlocal number
setlocal numberwidth=4
setlocal omnifunc=
setlocal path=
setlocal nopreserveindent
setlocal nopreviewwindow
setlocal quoteescape=\\
setlocal noreadonly
setlocal norightleft
setlocal rightleftcmd=search
setlocal noscrollbind
setlocal shiftwidth=4
setlocal noshortname
setlocal nosmartindent
setlocal softtabstop=0
setlocal nospell
setlocal spellcapcheck=[.?!]\\_[\\])'\"\ \ ]\\+
setlocal spellfile=
setlocal spelllang=en
setlocal statusline=
setlocal suffixesadd=
setlocal swapfile
setlocal synmaxcol=3000
if &syntax != 'erlang'
setlocal syntax=erlang
endif
setlocal tabstop=4
setlocal tags=
setlocal textwidth=80
setlocal thesaurus=
setlocal nowinfixheight
setlocal nowinfixwidth
setlocal wrap
setlocal wrapmargin=0
silent! normal! zE
let s:l = 16 - ((14 * winheight(0) + 23) / 46)
if s:l < 1 | let s:l = 1 | endif
exe s:l
normal! zt
16
normal! 034l
let &so = s:so_save | let &siso = s:siso_save
doautoall SessionLoadPost
" vim: set ft=vim :
syntax on

BIN
db/test/DECISION_TAB.LOG
View File

Binary file not shown.

BIN
db/test/LATEST.LOG
View File

Binary file not shown.

BIN
db/test/id_counter.DCD
View File

Binary file not shown.

BIN
db/test/id_counter.DCL
View File

Binary file not shown.

BIN
db/test/ts_entry.DCD
View File

Binary file not shown.

BIN
db/test/ts_entry.DCL
View File

Binary file not shown.

24
doc/db_layer.rst Normal file
View File

@ -0,0 +1,24 @@
TimeStamper DB Layer
====================
The following modules make up the database layer:
* ``ts_user``: Interface to user data.
* ``ts_timeline``: Interface to timeline data.
* ``ts_entry``: Interface to timeline entry data.
* ``ts_ext_data``: Interface to extended data that can be set on different
records.
* ``ts_db_records``: Definition of data records.
The following modules and files are implementation details of the DB layer:
* ``id_counter``: Adds support for unique, sequential ID generation.
* ``ts_common``: Provides the implementation for any operations that are common
between the different interfaces.
Philosophy
----------
The database layer should abstract all database-specific code away from the
caller. Users of the DB layer should not have to think about transactions,
locking, etc.

10
doc/issues/desktop/0020fn5.rst Normal file
View File

@ -0,0 +1,10 @@
Add exclusion filter for entries.
=================================
Add a way for users to add regexes for entries to be ignored in the
display. Exclusions may be per-timeline or per-user.
========= ==========
Created: 2011-06-10
Resolved: YYYY-MM-DD
========= ==========

7
doc/issues/desktop/0021tn5.rst Normal file
View File

@ -0,0 +1,7 @@
Constrain width of notes fields to width of mark.
=================================================
========= ==========
Created: 2011-06-10
Resolved: YYYY-MM-DD
====================

1
src/timestamper.erl
View File

@ -13,4 +13,5 @@ create_tables(Nodes) ->
{atomic, ok} = ts_user:create_table(TableOpts),
{atomic, ok} = ts_timeline:create_table(TableOpts),
{atomic, ok} = ts_entry:create_table(TableOpts),
{atomic, ok} = ts_ext_data:create_table(TableOpts),
ok.

1
src/timestamper_dev.erl
View File

@ -14,4 +14,5 @@ create_tables(Nodes) ->
{atomic, ok} = ts_user:create_table(TableOpts),
{atomic, ok} = ts_timeline:create_table(TableOpts),
{atomic, ok} = ts_entry:create_table(TableOpts),
{atomic, ok} = ts_ext_data:create_table(TableOpts),
ok.

45
src/ts_api.erl
View File

@ -204,11 +204,17 @@ get_user_summary(YArg, Username) ->
case ts_user:lookup(Username) of
no_record -> make_json_404(YArg);
User ->
EJSONUser = ts_json:record_to_ejson(User),
UserExtData - ts_ext_data:get_properties(Username),
EJSONUser = ts_json:record_to_ejson(User, UserExtData),
Timelines = ts_timeline:list(Username, 0, 100),
EJSONTimelines = {array,
lists:map(fun ts_json:record_to_ejson/1, Timelines)},
lists:map(
fun(Timeline) ->
ts_json:record_to_ejson(Timeline,
ts_ext_data:get_properties(Timeline#ts_timeline.ref))
end,
Timelines)},
JSONResp = json:encode({struct,
[{user, EJSONUser},
@ -245,7 +251,12 @@ list_timelines(YArg, Username) ->
Timelines = ts_timeline:list(Username, Start, Length),
% convert them all to their EJSON form
EJSONTimelines = {array, lists:map(fun ts_json:record_to_ejson/1, Timelines)},
EJSONTimelines = {array, lists:map(
fun (Timeline) ->
ts_json:record_to_ejson(Timeline,
ts_ext_data:get_properties(Timeline#ts_timeline.ref))
end,
Timelines)},
% create resposne
JSONResponse = json:encode(EJSONTimelines),
@ -268,12 +279,13 @@ put_timeline(YArg, Username, TimelineId) ->
%{struct, Fields} = EJSON,
% parse into a timeline record
TR = try ts_json:ejson_to_record_strict(
{TR, ExtData} =
try ts_json:ejson_to_record_strict(
#ts_timeline{ref={Username, TimelineId}}, EJSON)
catch _:InputError ->
catch throw:{InputError, _StackTrace} ->
error_logger:error_report("Bad input: ~p", [InputError]),
throw(make_json_400(YArg))
end,
throw(make_json_400(YArg, {request_error, InputError))
end,
ts_timeline:write(TR),
make_json_200(YArg, TR).
@ -321,7 +333,11 @@ list_entries(YArg, Username, TimelineId) ->
end,
EJSONEntries = {array, lists:map(
fun ts_json:record_to_ejson/1, Entries)},
fun (Entry) ->
ts_json:record_to_ejson(Entry,
ts_ext_data:get_properties(Entry#ts_entry.ref)),
end,
Entries)},
JSONResponse = json:encode(EJSONEntries),
@ -353,7 +369,11 @@ list_entries(YArg, Username, TimelineId) ->
end,
EJSONEntries = {array, lists:map(
fun ts_json:record_to_ejson/1, Entries)},
fun (Entry) ->
ts_json:record_to_ejson(Entry,
ts_ext_data:get_properties(Entry#ts_entry.ref)),
end,
Entries)},
JSONResponse = json:encode(EJSONEntries),
@ -374,16 +394,18 @@ post_entry(YArg, Username, TimelineId) ->
EJSON = parse_json_body(YArg),
% parse into ts_entry record
ER = try ts_json:ejson_to_record_strict(
{ER, ExtData} = try ts_json:ejson_to_record_strict(
#ts_entry{ref = {Username, TimelineId, undefined}}, EJSON)
catch _:InputError ->
error_logger:error_report("Bad input: ~p", [InputError]),
throw(make_json_400(YArg))
end,
%% TODO; should entries and their properties be created atomically?
case ts_entry:new(ER) of
% record created
{ok, CreatedRecord} ->
[{status, 201}, make_json_200(YArg, CreatedRecord)];
% will not create, record exists
@ -445,7 +467,8 @@ parse_json_body(YArg) ->
%% Create a JSON 200 response.
make_json_200(_YArg, Record) ->
JSONResponse = json:encode(ts_json:record_to_ejson(Record)),
RecordExtData = ts_ext_data:get_properties(element(2, Record)),
JSONResponse = json:encode(ts_json:record_to_ejson(Record, RecordExtData)),
{content, "application/json", JSONResponse}.
make_json_400(YArg) -> make_json_400(YArg, []).

52
src/ts_common.erl
View File

@ -1,32 +1,64 @@
-module(ts_common).
-export([new/1, update/1, update_record/2, list/3, order_datetimes/2]).
-export([new/1, new/2, update/1, update/2, update_record/2, do_set_ext_data/2,
list/3, order_datetimes/2]).
-include_lib("stdlib/include/qlc.hrl").
new(Record) ->
{atomic, Result} = mnesia:transaction(fun() -> do_new(Record) end),
Result.
new(Record, ExtData) when is_list(ExtData) ->
{atomic, Result} = mnesia:transaction(fun() ->
case do_new(Record) of
{error, Err} -> mnesia:abort({"Cannot create new record.", Err});
NewRecord -> case do_set_ext_data(Record, ExtData) of
ok -> NewRecord;
Error -> mnesia:abort(Error)
end
end
end),
Result.
% required to be wrapped in a transaction
do_new(Record) ->
Table = element(1, Record),
% check for existing record
case mnesia:dirty_read(Table, element(2, Record)) of
case mnesia:read(Table, element(2, Record)) of
% record exists
[ExistingRecord] -> {error, {record_exists, ExistingRecord}};
[] -> mnesia:dirty_write(Record)
[] -> mnesia:write(Record)
end.
update(Record) ->
{atomic, Result} = mnesia:transaction(fun() -> do_update(Record) end),
Result.
update(Record, ExtData) when is_list(ExtData) ->
{atomic, Result} = mnesia:transaction(fun() ->
case do_update(Record) of
{error, Err} -> mnesia:abort({"Cannot update record.", Err});
UpdatedRecord -> case do_set_ext_data(Record, ExtData) of
of -> UpdatedRecord;
Error -> mnesia:abort({"Cannot update record.", Error})
end
end
end),
Result.
do_update(Record) ->
Table = element(1, Record),
Key = element(2, Record),
% look for existing record
case mnesia:dirty_read(Table, Key) of
case mnesia:read(Table, Key) of
% record does not exist, cannot update
[] -> no_record;
% record does exist, update
[ExistingRecord] ->
mnesia:dirty_write(update_record(ExistingRecord, Record))
mnesia:write(update_record(ExistingRecord, Record))
end.
update_record(Record, UpdateData) ->
@ -66,6 +98,16 @@ list(Query, Start, Length) ->
end),
Result.
% should be wrapped in a transaction
do_set_ext_data(Record, ExtData) when is_list(ExtData) ->
Ref = element(2, Record),
lists:foreach(
fun({Key, Val}) ->
{atomic, ok} = ts_ext_data:set_property(Ref, Key, Val)
end,
ExtData)
ok.
% This is somewhat ridiculous.
order_datetimes({{Y1, Mon1, D1}, {H1, Min1, S1}},
{{Y2, Mon2, D2}, {H2, Min2, S2}}) ->

22
src/ts_db_records.hrl
View File

@ -4,12 +4,21 @@
pwd_salt,
name,
email,
join_date,
ext_data = [] % other extensible data
join_date
}).
% ts_user.ext_data can be:
% [{last_timeline, TimelineId}]
% ts_user.ext_data format is
% [{key, val}, {key, val}]
%
% Valid key/value pairs are:
%
% * last_timeline/TimelineId
%
% list() representing a timeline owned by this user
%
% * entry_exclusions/EntryExclusions
%
% list() of regular expressions to be excluded from display, seperated by '|'
-record(ts_timeline, {
ref, % {username, timelineid}
@ -29,6 +38,11 @@
expires
}).
% extensible data for arbitrary entities
-record(ts_ext_data, {
ref, % {ref, key}: reference with item ref and property key
value % value
}).
%-record(ts_session, {
%session_id,
%expires,

20
src/ts_entry.erl
View File

@ -1,5 +1,6 @@
-module(ts_entry).
-export([create_table/1, new/1, update/1, write/1, delete/1, lookup/3, list_asc/3, list_desc/3]).
-export([create_table/1, new/1, new/2, update/1, update/2, write/1, delete/1,
lookup/3, list_asc/3, list_desc/3]).
-include("ts_db_records.hrl").
-include_lib("stdlib/include/qlc.hrl").
@ -10,15 +11,27 @@ create_table(TableOpts) ->
{type, ordered_set}, {index, [timestamp]}]).
new(ER = #ts_entry{}) ->
{atomic, NewRow} = mnesia:transaction(fun() -> do_new(ER) end),
{ok, NewRow}.
new(ER = #ts_entry{}, ExtData) when is_list(ExtData) ->
{atomic, NewRow} = mnesia:transaction(fun() ->
NewRow = do_new(ER),
ts_common:do_set_ext_data(ER, ExtData),
NewRow end),
NewRow.
do_new(ER = #ts_entry{}) ->
{Username, TimelineId, _} = ER#ts_entry.ref,
NextId = id_counter:next_counter(ts_entry_id),
NewRow = ER#ts_entry{ref = {Username, TimelineId, NextId}},
ok = mnesia:write(NewRow),
NewRow end),
{ok, NewRow}.
NewRow.
update(ER = #ts_entry{}) -> ts_common:update(ER).
update(ER = #ts_entry{}, ExtData) when is_list(ExtData) ->
ts_common:update(ER, ExtData).
write(ER = #ts_entry{}) -> mnesia:dirty_write(ER).
@ -30,7 +43,6 @@ lookup(Username, TimelineId, EntryId) ->
delete(ER = #ts_entry{}) -> mnesia:dirty_delete_object(ER).
list({Username, Timeline}, Start, Length, OrderFun)
when is_integer(Start) and is_integer(Length) ->

47
src/ts_ext_data.erl Normal file
View File

@ -0,0 +1,47 @@
-module(ts_ext_data).
-export([create_table/1, set_property/3, get_property/2, get_properties/1]).
-include("ts_db_records.hrl").
create_table(TableOpts) ->
mnesia:create_table(ts_entry,
TableOpts ++ [{attributes, record_info(fields, ts_ext_data)},
{type, ordered_set}]).
% set last timeline
set_property(Ref=#ts_user{}, last_timeline, LastTimelineId) ->
do_set_property(Ref, last_timeline, LastTimelineId);
% Set exclusion_list for a User account
set_property(Ref=#ts_user{}, entry_exclusions, ExclusionList) ->
do_set_property(Ref, entry_exclusions, string:join(ExclusionList, "|"));
% Set exclusion_list for a Timeline entry
set_property(Ref=#ts_timeline{}, entry_exclusions, ExclusionList) ->
do_set_property(Ref, entry_exclusions, string:join(ExclusionList, "|"));
set_property(Ref, Key, Value) ->
throw(io_lib:format("Property '~s' not available for a ~s record.",
[Key, element(1, Ref)]).
get_property(Ref, PropKey) ->
{atomic, Result} = mnesia:transaction(fun() ->
case mnesia:read(ts_ext_data, {Ref, PropKey}) of
[] -> not_set;
[Property] -> Property
end
end),
Result.
get_properties(Ref) ->
{atomic, Result} = mnesia:transaction(fun() ->
MatchHead = #ts_ext_data{ref = {Ref, '_'}, _='_'},
mnesia:select(ts_ext_data, [{MatchHead, [], ['$_']}])
end),
Result.
do_set_property(Ref, PropKey, Val) ->
{atomic, Result} = mnesia:transaction(fun() ->
mnesia:write(#ts_ext_data{ref = {Ref, PropKey}, value = Val}).
end),
Result.

138
src/ts_json.erl
View File

@ -1,24 +1,25 @@
-module(ts_json).
-export([encode_record/1, record_to_ejson/1, ejson_to_record/2, ejson_to_record_strict/2]).
-export([encode_record/1, record_to_ejson/2,
ejson_to_record/2, ejson_to_record/3,
ejson_to_record_strict/2, ejson_to_record_strict/3]).
-include("ts_db_records.hrl").
encode_record(Record) -> lists:flatten(json:encode(record_to_ejson(Record))).
% User JSON record structure:
% User JSON record required structure:
% {"id": "john_doe",
% "name": "John Doe",
% "email": "john.doe@example.com",
% "join_date": "2011-01-01T12:00.000Z",
% "ext_data": {"last_timeline", "personal"}}
% "join_date": "2011-01-01T12:00.000Z"}
record_to_ejson(Record=#ts_user{}) ->
record_to_ejson(Record=#ts_user{}, ExtData) ->
{struct, [
{id, Record#ts_user.username},
{name, Record#ts_user.name},
{email, Record#ts_user.email},
{join_date, encode_datetime(Record#ts_user.join_date)},
{ext_data, Record#ts_user.ext_data}]};
{join_date, encode_datetime(Record#ts_user.join_date)}] ++
ext_data_to_ejson(ExtData)};
% Timeline JSON record stucture:
% {"user_id": "john_doe",
@ -26,7 +27,7 @@ record_to_ejson(Record=#ts_user{}) ->
% "created": "2011-01-01T14:00.000Z",
% "description:"Personal time-tracking."}
record_to_ejson(Record=#ts_timeline{}) ->
record_to_ejson(Record=#ts_timeline{}, ExtData) ->
% pull out the username and timeline id
{Username, TimelineId} = Record#ts_timeline.ref,
@ -35,7 +36,8 @@ record_to_ejson(Record=#ts_timeline{}) ->
{user_id, Username},
{id, TimelineId},
{created, encode_datetime(Record#ts_timeline.created)},
{description, Record#ts_timeline.desc}]};
{description, Record#ts_timeline.desc}] ++
ext_data_to_ejson(ExtData)};
% Entry JSON record structure:
% {"user_id": "john_doe",
@ -45,7 +47,7 @@ record_to_ejson(Record=#ts_timeline{}) ->
% "mark": "Workout.",
% "notes": "First workout after a long break."}
record_to_ejson(Record=#ts_entry{}) ->
record_to_ejson(Record=#ts_entry{}, ExtData) ->
% pull out the username, timeline id, and entry id
{Username, TimelineId, EntryId} = Record#ts_entry.ref,
@ -59,68 +61,78 @@ record_to_ejson(Record=#ts_entry{}) ->
{id, EntryId},
{timestamp, encode_datetime(DateTime)},
{mark, Record#ts_entry.mark},
{notes, Record#ts_entry.notes}]}.
{notes, Record#ts_entry.notes}] ++
ext_data_to_ejson(ExtData)}.
encode_datetime({{Year, Month, Day}, {Hour, Minute, Second}}) ->
lists:flatten(io_lib:format("~4.10.0B-~2.10.0B-~2.10.0BT~2.10.0B:~2.10.0B:~2.10.0B.~3.10.0BZ",
[Year, Month, Day, Hour, Minute, Second, 000])).
ejson_to_record(Rec=#ts_timeline{}, EJSON) ->
{struct, Fields} = EJSON,
ext_data_to_ejson(ExtData) -> ext_data_to_ejson(ExtData, []).
Created = case lists:keyfind(created, 1, Fields) of
false -> undefined;
{created, CreatedVal} -> decode_datetime(CreatedVal)
end,
ext_data_to_ejson([], ExtData) -> ExtData;
Desc = case lists:keyfind(description, 1, Fields) of
false -> undefined;
{description, DescVal} -> DescVal
end,
ext_data_to_ejson([{Key, Value}|T], Acc) ->
case Key of
entry_exclusions -> ext_data_to_ejson(T, [{Key, {array, Value}}|Acc]);
_Other -> ext_data_to_json(T, [{Key, Value}|Acc])
end.
Rec#ts_timeline{
created = Created,
desc = Desc };
ejson_to_record(Empty, {struct, EJSONFields}) ->
construct_record(Empty, EJSONFields, []).
ejson_to_record(Rec=#ts_entry{}, EJSON) ->
{struct, Fields} = EJSON,
ejson_to_record(Empty, Ref, EJSON) ->
Constructed = ejson_to_record(Empty, EJSON),
setelement(2, Constructed, Ref).
Timestamp = case lists:keyfind(timestamp, 1, Fields) of
false -> undefined;
{timestamp, TSVal} -> calendar:datetime_to_gregorian_seconds(
decode_datetime(TSVal))
end,
ejson_to_record_strict(Empty=#ts_timeline{}, EJSON) ->
Constructed = ejson_to_record(Empty, EJSON),
case Constructed of
#ts_timeline{created = undefined} ->
throw("Missing timeline 'created' field.");
#ts_timeline{desc = undefined} ->
throw("Missing timeline 'description' field.");
_Other -> Constructed
end;
Mark = case lists:keyfind(mark, 1, Fields) of
false -> undefined;
{mark, MarkVal} -> MarkVal
end,
ejson_to_record_strict(Empty=#ts_entry{}, EJSON) ->
Constructed = ejson_to_record(Empty, EJSON),
case Constructed of
#ts_entry{timestamp = undefined} ->
throw("Missing timelne 'timestamp' field.");
#ts_entry{mark = undefined} ->
throw("Missing timeline 'mark' field.");
#ts_entry{notes = undefined} ->
throw("Missing timeline 'notes' field/");
_Other -> Constructed
end.
Notes = case lists:keyfind(notes, 1, Fields) of
false -> undefined;
{notes, NotesVal} -> NotesVal
end,
ejson_to_record_strict(Empty, Ref, EJSON) ->
Constructed = ejson_to_record_strict(Empty, EJSON),
setelement(2, Constructed, Ref).
Rec#ts_entry{
timestamp = Timestamp,
mark = Mark,
notes = Notes}.
construct_record(Timeline=#ts_timeline{}, [{Key, Val}|Fields], ExtData) ->
case Key of
created -> construct_record(
Timeline#ts_timeline{created = decode_datetime(Value)},
Fields, ExtData);
description -> construct_record(Timeline#ts_timeline{desc = Value},
Fields, ExtData);
Other -> construct_record(Timeline, Fields, [{Key, Value}|ExtData])
end;
ejson_to_record_strict(Rec=#ts_timeline{}, EJSON) ->
{struct, Fields} = EJSON,
construct_record(Entry=#ts_entry{}, [{Key, Value}|Fields], ExtData) ->
case Key of
timestamp -> construct_record(
Entry#ts_entry{timestamp = calendar:datetime_to_gregoraian_seconds(
decode_datetime(Value))},
Fields, ExtData);
mark -> construct_record(Entry#ts_entry{mark=Value}, Fields, ExtData);
notes -> construct_record(Entry#ts_entry{notes=Vale}, Fields, ExtData);
Other -> construct_record(Entry, Fields, [{Key, Value}|ExtData])
end;
Rec#ts_timeline{
created = decode_datetime(element(2, lists:keyfind(created, 1, Fields))),
desc = element(2, lists:keyfind(description, 1, Fields))};
ejson_to_record_strict(Rec=#ts_entry{}, EJSON) ->
{struct, Fields} = EJSON,
Rec#ts_entry{
timestamp = calendar:datetime_to_gregorian_seconds(decode_datetime(
element(2, lists:keyfind(timestamp, 1, Fields)))),
mark = element(2, lists:keyfind(mark, 1, Fields)),
notes = element(2, lists:keyfind(notes, 1, Fields))}.
construct_record(Record, [], ExtData) -> {Record, ExtData}.
decode_datetime(DateTimeString) ->
% TODO: catch badmatch and badarg on whole function
@ -144,3 +156,15 @@ decode_datetime(DateTimeString) ->
list_to_integer(SecondString)},
{Date, Time}.
ejson_to_ext_data(ExtData) -> ejson_to_ext_data(ExtData, []).
ejson_to_ext_data([], ExtData) -> ExtData;
ejson_to_ext_data([{Key, Value}|T], Acc) ->
case Key of
entry_exclusions -> ext_data_to_ejson(T, [{Key, element(2, Value)}|Acc]);
_Other -> ext_data_to_json(T, [{Key, Value}|Acc])
end.

14
src/ts_timeline.erl
View File

@ -1,5 +1,5 @@
-module(ts_timeline).
-export([create_table/1, new/1, update/1, write/1, lookup/2, list/3]).
-export([create_table/1, new/1, new/2, update/1, update/2, write/1, lookup/2, list/3]).
-include("ts_db_records.hrl").
-include_lib("stdlib/include/qlc.hrl").
@ -11,8 +11,14 @@ create_table(TableOpts) ->
new(TR = #ts_timeline{}) -> ts_common:new(TR).
new(TR = #ts_timeline{}, ExtData) when is_list(ExtData) ->
ts_common:new(TR, ExtData).
update(TR = #ts_timeline{}) -> ts_common:update(TR).
update(TR = #ts_timeline{}, ExtData) when is_list(ExtData) ->
ts_common:update(TR, ExtData).
write(TR = #ts_timeline{}) -> mnesia:dirty_write(TR).
lookup(Username, TimelineId) ->
@ -22,8 +28,6 @@ lookup(Username, TimelineId) ->
end.
list(Username, Start, Length) ->
{atomic, Timelines} = mnesia:transaction(fun() ->
MatchHead = #ts_timeline{ref = {Username, '_'}, _='_'},
mnesia:select(ts_timeline, [{MatchHead, [], ['$_']}])
end),
MatchHead = #ts_timeline{ref = {Username, '_'}, _='_'},
mnesia:dirty_select(ts_timeline, [{MatchHead, [], ['$_']}]),
lists:sublist(Timelines, Start + 1, Length).

36
src/ts_user.erl
View File

@ -1,6 +1,6 @@
-module(ts_user).
%-export([create_table/1, new/1, update/1, lookup/1, list/2]).
-compile(export_all).
-export([create_table/1, new/1, new/2, update/1, update/2, lookup/1, list/2,
check_credentials/2]).
-include("ts_db_records.hrl").
-include_lib("stdlib/include/qlc.hrl").
@ -12,13 +12,35 @@ create_table(TableOpts) ->
% expects the password in clear
new(UR = #ts_user{}) ->
case mnesia:dirty_read(ts_user, UR#ts_user.username) of
{atomic, Result} = mnesia:transaction(fun() -> do_new(UR) end),
Result.
new(UR = #ts_user{}, ExtData) ->
{atomic, Result} = mnesia:transaction(fun() ->
NewUser = do_new(UR),
ts_common:do_set_ext_data(UR, ExtData),
NewUser end),
Result.
do_new(UR = #ts_user{}) ->
case mnesia:read(ts_user, UR#ts_user.username) of
[ExistingRecord] -> {error, {record_exists, ExistingRecord}};
[] -> mnesia:dirty_write(hash_input_record(UR))
[] -> mnesia:write(hash_input_record(UR))
end.
update(UR = #ts_user{}) ->
case mnesia:dirty_read(ts_user, UR#ts_user.username) of
{atomic, Result} = mnesia:transaction(fun() -> do_update(UR) end),
Result.
update(UR = #ts_user{}, ExtData) ->
{atomic, Result} = mnesia:transaction(fun() ->
UpdatedUser = do_update(UR),
ts_common:do_set_ext_data(UR, ExtData)
UpdatedUser end),
Result.
do_update(UR = #ts_user{}) ->
case mnesia:read(ts_user, UR#ts_user.username) of
[] -> no_record;
[Record] ->
UpdatedRecord = ts_common:update_record(Record, UR),
@ -26,14 +48,14 @@ update(UR = #ts_user{}) ->
undefined -> UpdatedRecord;
_Password -> hash_input_record(UpdatedRecord)
end,
mnesia:dirty_write(HashedRecord)
mnesia:write(HashedRecord)
end.
lookup(Username) ->
case mnesia:dirty_read(ts_user, Username) of
[] -> no_record;
[User] -> User
end.
end.
list(Start, Length) -> ts_common:list(ts_user, Start, Length).