Skip to content

TokenRunner updates

Currently the token runner pulls all AceTokens for a bag into memory. This can easily make the thread run out of heap space (tokens include the proof) if the bag has a large amount of files (>100k or so). It adds a little bit more strain to the db at the cost of lower memory utilization.

Steps include:

  • Adding queries to the TokenRepository for count/filenames.
    • May need an extra class for the filename query.
  • Using the count when doing the comparison
  • When writing tokens, only pulling a partial list of token at a time.