TokenRunner updates
Currently the token runner pulls all AceTokens for a bag into memory. This can easily make the thread run out of heap space (tokens include the proof) if the bag has a large amount of files (>100k or so). It adds a little bit more strain to the db at the cost of lower memory utilization.
Steps include:
- Adding queries to the TokenRepository for count/filenames.
- May need an extra class for the filename query.
- Using the count when doing the comparison
- When writing tokens, only pulling a partial list of token at a time.