Merge Tokenizer mq

Closes #95 (closed)

Should rebase some of the commits first to clean up the commit log before merging. Going to try that out in the merge itself rather than the branch.